Soft Robotics Finds Its Reach: Navigating Clutter with Hybrid Manipulators

by

Author: Denis Avetisyan

A new framework combines vision and control to enable robotic arms – blending rigid and soft components – to reliably grasp objects in complex, real-world environments.

A system integrates goal detection, three-dimensional reconstruction-enhanced by occupancy grids to map traversable space-and shape-informed path planning to enable a learned controller for hybrid manipulators, facilitating real-time reaching and control within complex environments and allowing actuation to any arbitrary pose.
A system integrates goal detection, three-dimensional reconstruction-enhanced by occupancy grids to map traversable space-and shape-informed path planning to enable a learned controller for hybrid manipulators, facilitating real-time reaching and control within complex environments and allowing actuation to any arbitrary pose.

This work presents HyReach, a vision-guided system for hybrid rigid-soft manipulators combining 3D reconstruction, shape-informed path planning, and learning-based control for robust reaching in unseen cluttered environments.

Robotic manipulation in real-world settings demands adaptability that traditional rigid systems often lack. This is addressed in ‘HyReach: Vision-Guided Hybrid Manipulator Reaching in Unseen Cluttered Environments’, which introduces a novel framework for hybrid rigid-soft manipulators capable of robust object reaching in previously unseen and cluttered environments. By integrating multi-view 3D reconstruction, shape-informed path planning, and learning-based control, HyReach achieves consistent performance with errors below 2cm without environment-specific retraining. Could this approach pave the way for more versatile and reliable robotic solutions in unstructured spaces like homes and warehouses?

The Limits of Conventional Design

Conventional robotic manipulators, often engineered with fixed geometries and pre-programmed movements, frequently falter when confronted with the unpredictable nature of real-world settings. These systems are typically reliant on meticulously mapped environments and predictable object placements; deviations from these controlled parameters – such as unexpected obstacles, deformable objects, or variations in lighting – can lead to failures in grasping, assembly, or navigation. This rigidity stems from the difficulty of creating actuators and control algorithms that can simultaneously manage a wide range of forces, positions, and orientations needed for robust manipulation in unstructured spaces. Consequently, achieving truly adaptable robotic manipulation requires a departure from these traditional designs, embracing strategies that prioritize sensing, learning, and compliance to overcome the limitations of pre-defined motions and static models.

Conventional robotic systems frequently falter when confronted with the unpredictability of real-world environments because they depend heavily on meticulously constructed models and rigidly controlled conditions. These robots are typically programmed assuming a static, predictable world – a scenario rarely encountered outside of laboratory settings. The presence of clutter, variations in object properties like weight or texture, and unexpected disturbances significantly degrade performance, as even minor deviations from the pre-programmed model can lead to failed grasps or collisions. This reliance on precision creates a fundamental limitation, hindering the deployment of robots in dynamic spaces such as homes, warehouses, or disaster zones where adaptability, rather than strict adherence to a plan, is paramount for successful operation.

Current robotic systems, largely predicated on pre-programmed movements within static landscapes, are proving insufficient for the demands of truly dynamic environments. A paradigm shift in robotic design is therefore essential, moving beyond rigid architectures and embracing adaptability as a core principle. This necessitates innovations in areas like soft robotics, utilizing compliant materials to allow for greater flexibility and resilience, and advancements in sensor integration, enabling robots to perceive and react to unforeseen obstacles or changes in their surroundings. Furthermore, algorithms focusing on reinforcement learning and imitation learning are crucial, allowing robots to learn through experience and replicate complex manipulation tasks without explicit programming. Ultimately, the future of robotic manipulation hinges on creating systems that don’t just operate in the unknown, but actively learn and adapt within it.

Using a hybrid rigid-soft continuum arm with a camera and magnetic sensor for pose tracking, our system successfully navigates cluttered environments to reach target objects.
Using a hybrid rigid-soft continuum arm with a camera and magnetic sensor for pose tracking, our system successfully navigates cluttered environments to reach target objects.

A Synergistic Hybrid Architecture

The Hybrid Manipulator System utilizes a combination of robotic technologies: a 6-Degree-of-Freedom (6-DoF) rigid manipulator and a B3 soft continuum arm. The 6-DoF rigid manipulator provides precise and repeatable movements for tasks requiring accuracy and force control. Complementing this is the B3 soft continuum arm, which is characterized by its inherent compliance and ability to bend in multiple directions without discrete joints. This architectural integration allows the system to achieve both the precision traditionally associated with rigid robots and the adaptability and dexterity offered by soft robotics, enabling operation in complex and constrained environments.

The Hybrid Manipulator System’s architecture allows for synergistic operation between the rigid 6-DoF manipulator and the flexible B3 soft continuum arm. The rigid component provides precise, high-force control for initial positioning and navigating to the general vicinity of a target. Subsequently, the soft arm facilitates access to confined spaces and enables compliant grasping of delicate or irregularly shaped objects. This combination minimizes the risk of damage during contact and allows manipulation of objects with varying fragility, exceeding the capabilities of either a purely rigid or purely soft robotic system. The resulting dexterity is particularly advantageous in unstructured environments where precise initial positioning is followed by delicate, adaptable manipulation.

Traditional robotic systems, typically employing solely rigid or solely soft construction, exhibit limitations when operating in unstructured environments. Rigid robots, while providing precise and repeatable movements, struggle with complex geometries and delicate object manipulation due to their limited adaptability and potential for damage. Conversely, soft robots offer enhanced compliance and adaptability but often lack the precision and load-bearing capacity required for certain tasks. The Hybrid Manipulator System addresses these shortcomings by integrating both technologies; the rigid component provides positional accuracy and force, while the soft component enables compliant navigation around obstacles and secure grasping of fragile items, effectively extending operational capabilities beyond the scope of either technology alone.

This fully learned hybrid arm controller achieves closed-loop control to an arbitrary pose by directly mapping a start pose [latex] p_{s} [/latex] and relative goal pose [latex] p_{rel} [/latex] to actuations, eliminating the need for explicit hybrid system modeling.
This fully learned hybrid arm controller achieves closed-loop control to an arbitrary pose by directly mapping a start pose [latex] p_{s} [/latex] and relative goal pose [latex] p_{rel} [/latex] to actuations, eliminating the need for explicit hybrid system modeling.

Perceiving and Planning for Deformable Movement

Environmental perception is achieved through multi-view reconstruction, employing the Mast3r and YOLO-World frameworks to create a detailed Occupancy Grid. Mast3r facilitates the reconstruction of 3D environments from multiple camera views, while YOLO-World provides object detection and segmentation data which is integrated into the grid. The resulting Occupancy Grid represents the workspace as a volumetric map, where each cell indicates the probability of being occupied by an obstacle. This grid-based representation provides the robot with crucial spatial awareness, enabling informed path planning and collision avoidance by quantifying free and occupied space within its operational environment.

Shape-informed path planning for the B3 Soft Continuum Arm utilizes a detailed environmental understanding, derived from multi-view reconstruction, in conjunction with the Constant Curvature (CC) Model. This approach ensures generated trajectories are both collision-free and physically feasible for the soft robot. The CC Model constrains path planning to curves with constant curvature, simplifying the kinematic calculations required to determine if a given trajectory can be executed by the arm without exceeding its physical limitations or encountering obstacles. By integrating environmental data with the CC Model, the system proactively avoids collisions and produces smooth, executable paths tailored to the unique characteristics of a soft, deformable robot.

The implementation of Rapidly-exploring Random Trees (RRT) in conjunction with the Constant Curvature (CC) model facilitates efficient path planning for the B3 Soft Continuum Arm by explicitly addressing its deformability. RRT* provides a sampling-based approach to quickly explore the configuration space, while the CC model constrains planned paths to adhere to the arm’s kinematic limitations and ensure feasibility. Path generation is guided by a defined Collision Threshold; trajectories are iteratively refined to maintain a safe distance from obstacles as determined by this threshold, preventing collisions and enabling robust navigation in complex environments. This integrated approach allows for the generation of smooth, collision-free paths that account for the unique challenges posed by soft robotic manipulation.

The hybrid arm successfully navigated four experimental setups-[latex]No~Obstacles[/latex], [latex]Obstacles[/latex], [latex]Clutter[/latex], and [latex]Hole[/latex]-as demonstrated by representative views from a single test run for each, ultimately reaching the designated goal object.
The hybrid arm successfully navigated four experimental setups-[latex]No~Obstacles[/latex], [latex]Obstacles[/latex], [latex]Clutter[/latex], and [latex]Hole[/latex]-as demonstrated by representative views from a single test run for each, ultimately reaching the designated goal object.

Robust Operation in Unstructured Environments

A learning-based controller, guided by real-time occupancy grid data, allows a hybrid robotic manipulator to achieve stable and precise movements toward target positions. This controller doesn’t rely on pre-programmed trajectories, but instead learns to adapt its actions based on the perceived environment. The occupancy grid provides a representation of free and occupied space, enabling the controller to plan paths that avoid collisions and maintain balance even as the environment changes. Through this learned adaptation, the manipulator exhibits enhanced robustness, consistently reaching desired poses with minimal error and maintaining stability throughout the actuation process – a crucial capability for operation in dynamic, real-world scenarios.

The robotic system exhibits a significant advancement in open-world reaching capabilities, consistently achieving high success rates even when confronted with unfamiliar surroundings. Testing across a range of environments – including those with no obstacles, moderate clutter, deliberate obstructions, and challenging holes – revealed an overall success rate peaking at 90.9%. This performance isn’t merely about completing the reach; the system also maintains impressive precision, consistently achieving mean translation errors of under 3 centimeters across all tested scenarios. These results demonstrate the system’s robust adaptability and its potential for deployment in dynamic, real-world applications where pre-programmed solutions are insufficient.

Rigorous testing across varied environments demonstrates the robustness of the reaching system. Performance benchmarks reveal a high success rate of 90.9% in both unobstructed spaces and cluttered scenes, indicating effective navigation and manipulation even with environmental complexity. While performance decreased to 75% when navigating around obstacles and 54.5% in the challenging ‘Hole’ setup – requiring precise maneuvering – the system maintained commendable accuracy. Mean translation errors remained consistently low, staying below 2 centimeters in the ‘No Obstacles’, ‘Obstacles’, and ‘Clutter’ scenarios and under 3 centimeters when reaching through the ‘Hole’, highlighting the controller’s precision and stability throughout the open-world reaching tasks.

The presented research on HyReach embodies a holistic approach to robotic manipulation, mirroring the interconnectedness of complex systems. This work doesn’t simply address reaching in cluttered spaces; it meticulously integrates vision, reconstruction, planning, and control as a unified framework. As Marvin Minsky aptly stated, “You can’t solve problems by just adding more stuff.” HyReach demonstrates this principle by prioritizing a streamlined, integrated architecture over brute-force solutions. The system’s ability to navigate unseen environments highlights how understanding the entire system – from sensor input to actuator output – is paramount to achieving robust performance. The focus on shape-informed path planning, a core concept of the study, exemplifies this emphasis on system-wide coherence.

What’s Next?

The demonstrated capacity for a hybrid manipulator to navigate previously unseen clutter represents a step, but not a destination. Current approaches, even those leveraging vision and learning, remain fundamentally reactive. The system accurately responds to the environment, but lacks the predictive capability inherent in truly intelligent action. Future work must address the chasm between reconstruction and anticipation – moving beyond mapping static obstacles to modeling dynamic interactions and potential failures. The true challenge lies not in avoiding collisions, but in gracefully recovering from inevitable perturbations.

Furthermore, the inherent limitations of shape-informed planning become readily apparent when considering increasingly complex geometries or deformable objects. While current methods excel in relatively structured environments, they falter when confronted with the ambiguity of the truly disordered. A deeper integration of material properties – not just geometric shape, but elasticity, friction, and compliance – will be crucial. The elegance of a continuum robot is quickly diminished if its control system cannot account for its own inherent flexibility.

Ultimately, this work serves as a reminder that robotics, despite its rapid progress, remains fundamentally an exercise in applied pragmatism. Good architecture is invisible until it breaks, and only then is the true cost of decisions visible.

Original article: https://arxiv.org/pdf/2603.21421.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

See also:

2026-03-24 13:40