Safeguarding Collaborative Robots with AI

by

Author: Denis Avetisyan

A new approach uses large language models to formalize safety protocols and redundant systems to ensure reliable operation in human-robot workcells.

The deployment of a deterministic safety agent, guided by a large language model, demonstrates a pathway toward graceful system degradation on a redundant dual-RK3588 SoC platform-a configuration designed not to resist inevitable decay, but to manage it with predictable control.
The deployment of a deterministic safety agent, guided by a large language model, demonstrates a pathway toward graceful system degradation on a redundant dual-RK3588 SoC platform-a configuration designed not to resist inevitable decay, but to manage it with predictable control.

This paper details an ISO-compliant, perception-compute-control architecture featuring an LLM-guided safety agent for deterministic edge robotics.

Achieving functional safety in increasingly collaborative robotic systems presents a fundamental challenge, as the probabilistic nature of AI perception clashes with the deterministic requirements of industrial standards. This paper introduces an ‘LLM-Guided Safety Agent for Edge Robotics with an ISO-Compliant Perception-Compute-Control Architecture’ that bridges this gap by translating natural language safety regulations into executable predicates deployed on a redundant, low-latency edge computing platform. Our approach demonstrates a practical pathway toward ISO 13849 Category 3 and PL d compliance using cost-effective hardware and a symmetric dual-modular redundancy design. Could this architecture pave the way for more robust and reliable deployment of safety-critical AI in real-world human-robot interaction scenarios?

The Inevitable Dance: Beyond Traditional Safety Boundaries

Conventional safety standards, such as ISO 13849-1, were initially designed for stationary industrial machinery operating within predefined, predictable parameters. However, the rise of collaborative robots-or ‘cobots’-working alongside humans introduces a level of dynamism and unpredictability these standards struggle to address. These cobots don’t simply perform repetitive tasks in fixed locations; they adapt, learn, and share workspaces with people whose movements are inherently variable. Consequently, relying solely on traditional hazard avoidance techniques-like emergency stops and physical barriers-proves insufficient. A robot halting upon any human proximity, while safe in principle, drastically limits the potential for true collaboration and productivity. The limitations stem from a fundamental mismatch: existing standards prioritize preventing any contact, whereas effective human-robot teamwork requires nuanced risk assessment that accounts for intent, speed, and the nature of the interaction itself.

Conventional safety protocols prioritize hazard avoidance – stopping a robot when a human enters its workspace. However, truly collaborative scenarios demand a more nuanced understanding of safety, one that extends beyond simply preventing collisions. A comprehensive definition must also consider target integrity – ensuring the human operator or workpiece isn’t inadvertently harmed by the robot’s actions, even during intended operation. Crucially, behavioral dynamics are central; safety isn’t a static state, but rather a continuous assessment of how the robot and human adapt to each other’s movements and intentions. This means evaluating not just what the robot is doing, but how it’s doing it – its speed, trajectory, and responsiveness – to guarantee a safe and productive partnership throughout the entire interaction.

Effective risk assessment in collaborative robotics demands a move beyond simplistic hazard identification towards a nuanced, multi-tiered semantic hierarchy. This framework organizes potential risks not merely by their presence, but by their meaning and potential consequences, creating layers of understanding. At the foundational level, basic hazards – such as collisions or trapped limbs – are defined. Above this, the hierarchy considers the integrity of the target – is it a fragile component, a human worker, or a robust structure? – influencing the severity of impact. Finally, the system analyzes behavioral dynamics, factoring in robot speed, trajectory predictability, and human responsiveness to accurately gauge the probability of a harmful interaction. This structured approach allows for more precise risk mitigation strategies, tailoring protective measures to the specific context and minimizing unnecessary restrictions on collaborative workflows.

The safety agent employs a hierarchical framework-starting with LLM-based constraint extraction ([latex]CA, CBC_{A}, C_{B}[/latex]), progressing to multidimensional perception alignment incorporating spatial mapping ([latex]S_{safe}[/latex]) and temporal WCET analysis ([latex]T_{stop}[/latex]), and culminating in a Perception-Compute-Control architecture-to ensure safe operation.
The safety agent employs a hierarchical framework-starting with LLM-based constraint extraction ([latex]CA, CBC_{A}, C_{B}[/latex]), progressing to multidimensional perception alignment incorporating spatial mapping ([latex]S_{safe}[/latex]) and temporal WCET analysis ([latex]T_{stop}[/latex]), and culminating in a Perception-Compute-Control architecture-to ensure safe operation.

Formalizing Intent: Guiding Safety with Language

LLM-Guided Formalization utilizes large language models to convert qualitative safety standards, exemplified by ISO 13849-1 which governs the safety of machinery, into a set of formal predicates. These predicates are logical statements expressed in a machine-readable format, allowing for automated verification and integration into robotic control systems. The process involves parsing the natural language text of the standard, identifying safety-critical requirements, and translating these into logical expressions defining permissible and impermissible system states. This translation enables quantitative analysis of safety compliance and facilitates the creation of verifiable safety constraints within a robotic application, moving beyond purely descriptive compliance documentation.

Direct integration of formally defined safety requirements into robotic control systems enables proactive risk mitigation by shifting from reactive safety measures to preventative ones. This is achieved by translating high-level safety standards into executable predicates that the control system continuously monitors during operation. Instead of responding to hazardous situations as they arise, the system actively prevents them by verifying that predefined safety conditions are met before, during, and after task execution. This approach facilitates real-time hazard detection and allows for automated adjustments to robot behavior or system shutdown to prevent potential harm, increasing overall system reliability and reducing the likelihood of incidents.

Large Language Models (LLMs) facilitate the translation of natural language safety standards into formal predicates usable by automated systems. Traditionally, implementing safety requirements like those defined in ISO 13849-1 requires manual interpretation and coding, a process prone to error and scalability issues. LLMs, through their capacity for semantic understanding and logical inference, can parse these standards and generate corresponding logical expressions, such as those used in formal verification or runtime monitoring. This automated translation reduces implementation time, minimizes the risk of misinterpretation, and enables direct integration of safety constraints into robotic control architectures, allowing for proactive hazard avoidance and system validation.

Building Resilience: The Logic of Redundancy

Dual-Modular Redundancy (DMR) is a fault-tolerant design technique that utilizes two identical hardware modules operating in parallel. Each module independently processes the same input data and produces an output. A comparison mechanism then evaluates the outputs of both modules; discrepancies indicate a fault within one of the modules. This allows the system to detect and isolate failures without interrupting operation, relying on the functional module to continue processing. The redundancy inherent in DMR significantly enhances system reliability and safety by mitigating the impact of single-point failures, and is particularly applicable in critical applications where continuous availability is paramount.

Dual-Modular Redundancy (DMR) relies on Parallel Independent Execution (PIE) where identical hardware modules process the same data simultaneously and independently, allowing for comparison of results to detect discrepancies. Comprehensive hardware monitoring is achieved through techniques like Hardware Probing using Analog-to-Digital Converters (ADC) to assess voltage and current levels of critical components, and UART Heartbeats, which are periodic signals transmitted between modules to confirm operational status. These mechanisms enable the system to identify and isolate failing components without interrupting overall functionality, bolstering system integrity and availability.

Dual-Modular Redundancy (DMR) achieves continuous operation by deploying two identical processing channels that operate in parallel. Each channel independently executes the same functions, and outputs are compared. Discrepancies indicate a fault within one channel, triggering a switch to the functioning channel, thereby maintaining system operation without interruption. This redundancy is particularly crucial in collaborative environments – such as robotics or automated control systems – where a single point of failure could lead to hazardous outcomes or significant operational downtime. The architecture mitigates risks associated with transient hardware faults, software errors, and even malicious interference, providing a demonstrable increase in system safety and availability.

The integrated PPC system employs a symmetrically redundant, dual-node architecture centered around a Rockchip RK3588 SoC to enable a deterministic hardware execution pipeline from raw sensor data through external I/O.
The integrated PPC system employs a symmetrically redundant, dual-node architecture centered around a Rockchip RK3588 SoC to enable a deterministic hardware execution pipeline from raw sensor data through external I/O.

A System in Equilibrium: PPC Architecture in Action

The system’s core relies on a Perception-Compute-Control (PPC) architecture, strategically merging robust hardware redundancy with a responsive control system. This is achieved through the integration of a Dual-Module Redundancy (DMR) setup, which provides backup functionality in case of component failure, and a high-performance, low-latency control loop driven by the Rockchip RK3588 system-on-chip. This powerful combination allows for continuous monitoring and swift reaction to changing conditions, forming the foundation for a safe and reliable operational framework. The RK3588’s processing capabilities are central to interpreting sensory input and executing control commands with minimal delay, ensuring the system remains stable and predictable even in dynamic environments.

The system’s architecture is fundamentally built upon Functional Safety principles, prioritizing hazard identification and mitigation through a continuously active monitoring and control loop. This proactive approach moves beyond reactive safety measures by constantly assessing potential risks and implementing corrective actions before incidents occur. Central to this capability is a remarkably efficient execution time; even under the most demanding conditions, the system achieves a worst-case execution time (WCET) of approximately 65 milliseconds. This rapid processing speed is critical for ensuring timely responses to dynamic situations and maintaining a safe operational environment, especially in applications where swift decision-making is paramount for preventing harm or damage.

The system’s capacity for swift decision-making hinges on its Neural Processing Unit (NPU), achieving an inference latency of just 25 milliseconds. This remarkably quick processing speed – consistently delivered with a minimal standard deviation of 0.64ms – allows for near-instantaneous reactions to changing conditions. Such low latency is critical for applications demanding real-time responsiveness, effectively minimizing delays between perception of an event and the initiation of a corrective action. The NPU’s performance directly contributes to a safer and more reliable operational environment, particularly within dynamic settings like human-robot collaboration where prompt and predictable responses are paramount.

The system’s robust safety framework is underscored by its exceptionally swift failure detection capabilities. Critical components are continuously monitored, enabling the identification of anomalies within milliseconds; a stalled Neural Processing Unit (NPU) is flagged in just 2.04ms, while power fluctuations – such as brownouts – are detected in 38.45ms. Furthermore, the system reliably identifies communication disruptions, registering a heartbeat loss in 51.87ms, and can pinpoint sensor malfunctions in up to 2010.45ms. These rapid response times, achieved through dedicated hardware and software optimization, are fundamental to maintaining operational integrity and ensuring a safe and predictable environment, particularly in applications demanding high reliability and real-time performance.

The architecture’s commitment to established safety standards, notably ISO 15066, is central to enabling dependable human-robot collaboration. This adherence isn’t merely about compliance; it actively builds confidence in scenarios where humans and robots share workspaces. By rigorously addressing potential hazards and implementing continuous monitoring, the system minimizes risks and allows for more fluid, intuitive interaction. Consequently, collaborative processes become demonstrably safer, fostering trust between human workers and robotic assistants, and ultimately boosting overall efficiency by optimizing task allocation and reducing downtime associated with safety concerns.

Performance was evaluated across three scenarios-a clear industrial workspace, a partially obscured environment, and a complex multi-worker layout-using danger zones [latex]RR_{ROI}[/latex] and detected human subjects to assess the system's robustness.
Performance was evaluated across three scenarios-a clear industrial workspace, a partially obscured environment, and a complex multi-worker layout-using danger zones [latex]RR_{ROI}[/latex] and detected human subjects to assess the system’s robustness.

The pursuit of robust, adaptable systems-particularly those operating at the edge-demands acknowledging inherent decay. This work, detailing an LLM-guided safety agent within a redundant Perception-Compute-Control architecture, implicitly recognizes this truth. Versioning, in this context, isn’t merely an engineering practice; it’s a form of memory, a deliberate attempt to capture and refine safety protocols over time. As John von Neumann observed, “In the past, the problem was to build things. Now the problem is to prevent them from being built.” Preventing unintended consequences in dynamic industrial environments requires a continuous cycle of formalization, verification, and adaptation-a process elegantly addressed by the proposed safety agent and its commitment to deterministic operation, even amidst the inherent unpredictability of large language models.

What’s Next?

The presented architecture, while addressing critical redundancies for deterministic operation, merely postpones the inevitable cascade of errors inherent in any complex system. The integration of large language models into a safety-critical framework is not a solution, but a shift in the error profile-from predictable hardware failures to the more subtle, and arguably less tractable, failings of semantic interpretation. Future work must confront the fact that formalizing safety requirements in natural language is, itself, an act of approximation, introducing a new vector for unexpected behavior.

The true metric isn’t adherence to ISO compliance, but the rate of graceful degradation. The system will fail-the question becomes whether those failures offer opportunities for refinement, or accelerate the path to irrecoverable states. Further investigation should focus on quantifying the ‘semantic drift’ within the LLM’s reasoning, and developing mechanisms for runtime verification of its safety assertions.

Ultimately, this work represents a step toward more adaptable robotic systems, but adaptability is not synonymous with safety. The field must move beyond simply detecting failures, and embrace the concept of anticipatory resilience-designing systems that not only tolerate errors, but actively learn from them, transforming incidents into steps toward maturity.

Original article: https://arxiv.org/pdf/2604.20193.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

See also:

2026-04-24 05:31