Abstract

Robotic dexterous manipulation requires continuously reconciling objectives and constraints defined on heterogeneous geometric spaces: a robot controlled on an ℝ⁷ configuration manifold may need to track end-effector poses on SE(3) while satisfying obstacle-avoidance margins in ℝ. We present Safe Pullback Bundle Dynamical Systems (SafePBDS), a geometrically consistent motion-generation framework that computes optimal, certifiably safe configuration-manifold accelerations from objectives and safety requirements defined on arbitrary task manifolds. SafePBDS builds on prior work that combines predefined task-manifold dynamical systems to produce autonomous motion. Its first innovation is a pullback control barrier function construction, which converts task-manifold safety conditions into linear constraints on configuration-manifold accelerations. The second innovation is a task-manifold action interface that allows a high-level policy to inject low-dimensional residual motions; zero input recovers the autonomous behavior, while safety is preserved under arbitrary inputs. This enables high-level policies to efficiently steer exploration while leaving precise motion generation to the autonomous behavior. We validate SafePBDS in simulation and on a 23-DOF Franka Panda–Allegro Hand platform. On dexterous grasping, SafePBDS achieves a 92.5% success rate across 20 household objects and 120 trials. By leveraging the action interface, the method can exclude any one of the four fingers during grasping using a one-dimensional action, achieving 94.4% 3-finger grasp success across 3 objects and 36 trials. The efficient planning and safety guarantee of SafePBDS also enables the first model-based, fully actuated palm-down in-hand reorientation, exceeding 360° of yaw rotation in both directions under varying object weight and wrist motion.

Method overview

SafePBDS extends Pullback Bundle Dynamical Systems (PBDS) with hard safety guarantees and an external action interface, while preserving its geometric consistency. At each control step, tasks defined on heterogeneous manifolds (an autonomous behavior, safety requirements, and high-level actions) are pulled back to the configuration manifold and composed into a single safe acceleration through quadratic programs (QPs).

Two innovations make this possible. Pullback control barrier functions (CBFs) convert task-manifold safety conditions into linear constraints on configuration-manifold accelerations by pulling the constraints back through smooth task maps; we derive both exponential (ECBF) and backstepping (BCBF) variants. A task-manifold action interface lets a higher-level policy inject inputs on selected task manifolds, with the guarantee that zero input recovers the autonomous behavior and that safety holds under any input. Two convex QPs are solved sequentially per step: the first computes the autonomous safe acceleration, and the second injects the action around it.

---

Simulation

We first validate the theoretical properties of SafePBDS in simulation: chart invariance, the effect of task-manifold geometry, recovery from unsafe states, hard safety under adversarial inputs, and behavior selection through the action interface.

Sphere (𝕊²) double integrator

A unit-mass point robot moves on the surface of a sphere. The autonomous behavior is an attractor with damping, and the safety task keeps the robot at least a fixed arclength away from an obstacle. The pullback ECBF is metric-independent, while the BCBF depends on the chosen task-manifold metric and keeps a larger clearance. Both formulations recover when started inside the unsafe set, and results are consistent across north- and south-pole charts. Through the action interface, equal-and-opposite tangential inputs yield solution paths in opposite homotopy classes around the obstacle. Adversarial inputs directed into the obstacle are clipped by the CBF, preserving safety.

7-DOF robot arm

We instantiate a 7-DOF arm in MuJoCo. With the obstacle-avoidance ECBF active, the full system safely deflects around a workspace obstacle while tracking an end-effector pose; the ablation without the obstacle CBF passes through it. Across 50 randomized orientation-tracking scenarios, the full system is safe in all 50 runs, whereas the ablation violates the obstacle constraint in 11 of 50. Using the action interface, a single transient scalar input on the first joint selects which side of the obstacle the arm passes (opposite homotopy classes) while the end effector converges to the same goal.

---

Hardware setup

All hardware experiments use a Franka Emika Panda 7-DOF arm and a Wonik Allegro 16-DOF hand, for a combined 23-DOF system. Runtime perception uses an Intel RealSense D435 with Segment Anything for segmentation and FoundationPose for 6-DOF pose tracking at 10 Hz, which updates an object pose in a MuJoCo model that the SafePBDS controller runs against. Object meshes are obtained by scanning with a LiDAR-equipped iPhone. The arm is controlled at 20 Hz with a joint-impedance controller while SafePBDS is active.

---

4-finger dexterous grasping

We apply SafePBDS to autonomous grasping of 20 household objects, spanning masses of 64–613 g and longest dimensions of approximately 6–29 cm, with six trials per object. After selecting and approaching a top-down wrist pose, SafePBDS takes over control of the full 23-DOF system and solves the PBDS QP at every control step. Per-finger fingertip-to-object distance controllers, together with an action that aligns the centroid of the four fingertips with the object's geometric center, drive the fingertips toward the object surface, while a force-closure ECBF is activated as the fingers approach. This guides the final squeeze to converge to a certified force-closed grasp without any object-specific tuning.

Two consecutive successful grasps per object, shown at 2× speed.

---

3-finger dexterous grasping

To showcase the flexibility of the framework, we grasp with one of the four fingers excluded using the action interface. The active force-closure ECBF is switched to the per-finger-excluded variant, and the excluded finger's distance action is set to a PD control law that moves the finger to a fixed distance away from the object.

Clips below are real-time (1× speed).

---

Palm-down in-hand reorientation

Finally, we apply SafePBDS to palm-down in-hand reorientation (IHR): rotating a grasped object about the downward-facing palm normal. This is far harder than the common palm-up setting, because the object must remain securely grasped throughout. An offline tree search over finger-gaiting states discovers feasible reorientation trajectories, each forward-simulated under SafePBDS so that a certified force-closure grasp is maintained at every step. The resulting open-loop plans are played back on hardware.

Using the same open-loop plan, the system is robust across three conditions, each shown below rotating both clockwise and counterclockwise (the first two conditions from two camera views). The full compilation is also in the demo video above.

All in-hand reorientation videos are real-time (1× speed).

---

Conclusion

SafePBDS is a geometric motion-generation framework that extends PBDS with task-manifold safety guarantees and an action interface. At each control step, two convex QPs are solved sequentially: the first defines the autonomous safe acceleration, and the second injects the action input around it. The resulting configuration-space acceleration is certifiably safe and recovers the autonomous task dynamics when the action input vanishes. On a 23-DOF Franka–Allegro system, SafePBDS attains 92.5% grasp success across 20 household objects and 94.4% for 3-finger grasps through simple action changes, and enables the first model-based robust palm-down in-hand reorientation, producing over 360° rotation in both directions under varying object weight and arm motion.

BibTeX

Upcoming