Hamilton-Jacobi Reachability for Hybrid Systems: Unified Goal-Driven Control with Safety Guarantees

AI summary

Overview

Research area: Formal safety verification and control synthesis for hybrid (continuous-discrete) dynamical systems, applied to robotics.

Technical level: Advanced. The paper builds on Hamilton-Jacobi (HJ) reachability theory, viscosity solutions to constrained Hamilton-Jacobi-Isaacs variational inequalities (HJI-VIs), and hybrid automata, and assumes familiarity with optimal control and dynamic programming.

Scope: The paper extends HJ reachability analysis to hybrid systems with controlled transitions, forced transitions, and state resets, then uses the resulting safe sets to build (i) a minimally invasive online safety filter and (ii) an offline reach-avoid controller that jointly guarantees safety and goal completion, validated on a quadrupedal robot.

What This Paper Is About

Robots that make and break contact (legged robots, jumping robots, manipulators that grasp) are naturally modeled as hybrid systems: continuous motion punctuated by discrete mode switches. Ensuring such systems are both safe and capable of completing tasks is hard because the controller must reason jointly about continuous inputs and discrete switching decisions. The authors extend Hamilton-Jacobi reachability — a formal method for continuous systems — to this hybrid setting, and then build two control mechanisms on top of it: one that filters unsafe actions from any existing policy, and one that synthesizes policies guaranteeing both safety and goal-reaching.

Key Contributions

1. Hybrid Least Restrictive Filter (hLRF). A real-time safety filter that wraps any nominal hybrid control policy and intervenes on both the continuous input and the discrete transition decisions only when necessary — i.e., only when the nominal choice would lead into the hybrid Backward Reachable Tube (hBRT) defined unsafe set.

2. Formal safety guarantee for the hLRF. A proof that the filter renders the hBRT-defined safe set forward invariant for any nominal hybrid controller, even in the presence of state resets, forced transitions, and mode-dependent dynamics — features absent from continuous-time safety filters.

3. Hybrid Backward Reach-Avoid Tubes (hBRATs). A new formulation that jointly encodes safety and goal-reaching in the hybrid setting, derived as a novel constrained HJI-VI whose sub-zero level set characterizes the set of states from which the system can reach a target while never entering a failure set.

4. Provably safe and goal-complete hybrid policies. Synthesis of both continuous and discrete control laws from the hBRAT solution, with validation on a real quadrupedal robot.

Main Findings

• Hybrid value function solves the reach-avoid problem. The solution of the newly derived constrained HJI-VI characterizes exactly the hBRAT: the set of hybrid states from which, for all disturbances, some control sequence reaches the goal without entering the failure set. The hBRT (safety-only) result is recovered as a special case.

• The hLRF preserves nominal behavior whenever safe. The filter only overrides the nominal policy when the value function evaluated at the post-transition state drops below zero, meaning the nominal discrete switch or continuous input would make failure unavoidable. Otherwise the nominal behavior passes through unmodified.

• Filter guarantee holds for arbitrary nominal controllers. Forward invariance of the safe set is proven independent of how the nominal policy was designed, making the filter a drop-in wrapper for learned, optimization-based, or hand-tuned controllers.

• Numerical algorithm is structurally similar to the classical continuous case. The proposed grid-based scheme propagates N value functions (one per discrete mode) in parallel and, after each time step, adjusts each mode's value by taking the minimum over available discrete transitions. The only differences from standard HJ reachability are the parallel propagation and this discrete adjustment step.

• Complexity scales quadratically in modes and exponentially in continuous state dimension. Total cost is O(T · N² · M^{n_x} / δ) for N modes, M grid points per dimension, n_x continuous states, horizon T, and time step δ. This makes the approach practical for low-dimensional hybrid systems but subject to the usual curse-of-dimensionality.

• Exact hybrid reachability is undecidable in general. The authors acknowledge that the algorithm computes an approximation and may not terminate for arbitrary hybrid dynamics, citing known undecidability results — yet report strong empirical performance on real hardware.

• Hardware validation succeeds. Experiments on a quadrupedal robot demonstrate both hybrid mode planning and safety-critical behavior, confirming the framework's practical viability beyond simulation.

Methodology in Plain English

The authors start from the standard idea of writing down a "value function" that scores every state of the system: negative values mean the state is good (a goal can be reached, or failure can be avoided), positive values mean it is bad. For ordinary continuous systems, this value function is the solution of a partial differential equation called the HJI-VI, solved backward in time from the goal or failure set.

Their extension handles the discrete side of hybrid systems by maintaining one such value function per discrete mode and, at every step, asking a simple question: "Is it better to stay in this mode, or to switch to a neighboring mode (possibly with a state reset)?" Taking the minimum over these options yields the updated value. Forced transitions — the switches the system must take when its continuous state leaves a mode's valid domain — are handled by removing the "stay put" option.

Building on this, the safety filter works at run time by first letting the nominal policy propose a discrete switch. If the resulting post-switch state has a value above zero (i.e., is unsafe), the filter substitutes the optimal safe switch instead. After the transition, the same check is applied to the continuous input. This is the "least restrictive" property: intervention happens only at the boundary of the safe set.

The reach-avoid formulation generalizes the value function further so that it simultaneously tracks "can I reach the goal?" and "can I avoid failure?", producing a single tube (the hBRAT) whose interior states admit a control strategy achieving both objectives.

Why This Matters

Impact on research. The paper closes a gap the authors themselves explicitly identified in their prior work: hybrid reachability could verify safety but could not synthesize goal-directed behavior. By unifying safety and liveness in a single value-based formulation, it moves hybrid HJ reachability from a verification tool to a control synthesis tool. It also connects the reachability literature to the hybrid control barrier function literature, which has largely focused on verifying candidate barriers rather than constructing global safe sets.

Real-world applications:

• Legged locomotion. Quadruped and bipedal robots that switch gaits, launch jumps, or land — precisely the hardware validated in this paper.
• Contact-rich manipulation. Robotic arms that make and break contact with objects, where grasping and releasing are discrete events embedded in continuous motion.
• Autonomous driving with mode switches. Vehicles that transition between platooning, lane-keeping, and emergency braking modes while remaining provably collision-free.
• Aerial and space robotics. Drones that transition between perching, hovering, and free flight, or spacecraft that switch between thruster configurations.

Industry relevance. The hLRF is attractive industrially because it is a wrapper: companies can keep their existing (possibly learned or hand-tuned) controllers and add a formally verified safety layer on top. This mirrors the deployment pattern already common in autonomous driving and industrial robotics, where a certified safety monitor supervises a performance-optimized planner. The approach is most immediately applicable to low-dimensional hybrid systems, where grid-based value function computation remains tractable.

Future Directions

• Scaling to higher dimensions. The exponential dependence on continuous state dimension limits the approach to relatively low-dimensional systems. Learning-based or approximate value function representations — already used in continuous HJ reachability — are the natural path forward.

• Handling adversarial forced transitions. The paper notes that forced transitions can be treated as worst-case (maximization instead of minimization), but does not fully develop control synthesis under genuinely adversarial switching. This is relevant when transitions are environmental rather than self-triggered.

• Guaranteeing termination of the numerical algorithm. Exact hybrid reachability is undecidable, so the current method is necessarily approximate. Characterizing tractable subclasses of hybrid systems with guaranteed convergence would strengthen the theoretical foundations.

• Extending to multi-agent hybrid systems. Many contact-rich robotic tasks (e.g., multi-robot manipulation) involve coupled hybrid dynamics across agents; the current formulation is single-agent.

• Tighter integration with learning. Combining the hLRF with learned nominal policies that are known to be near-optimal, so that safety filtering rarely activates, is an obvious practical direction.

Target Audience

This paper is aimed at researchers and graduate students in control theory, formal methods, and robotics — particularly those working on safety-critical control, reachability analysis, or legged locomotion. Practitioners building safety layers for hybrid robotic systems will find the hLRF directly actionable, while theorists will be more interested in the hBRAT derivation and the constrained HJI-VI. Readers need a working background in optimal control, dynamic programming, and hybrid automata; the paper is not accessible without that foundation.

Authors’ abstract

Hybrid dynamical systems provide a powerful modeling framework for robotic systems, particularly in contact-rich environments. However, ensuring safety and performance in such systems remains challenging due to the intricate coupling between continuous dynamics and discrete mode transitions. In this work, we extend classical Hamilton-Jacobi (HJ) reachability analysis, a formal verification method for continuous-time nonlinear systems, to hybrid dynamical systems. Our framework characterizes safe sets for hybrid systems through a generalized value function defined over both discrete and continuous states while accounting for control constraints and model uncertainty. We additionally provide a numerical algorithm to compute this value function. Building on these safe sets, we propose two different mechanisms to integrate performance objectives. First, we introduce a hybrid least-restrictive safety filter that intervenes on both the discrete and continuous components of a nominal controller only when necessary to avoid unsafe states, thereby preserving nominal behavior whenever possible. Second, we formulate and compute hybrid backward reach-avoid tubes, enabling the simultaneous enforcement of safety and goal-reaching behavior, an extension not previously addressed within hybrid HJ reachability. This enables the synthesis of continuous and discrete control policies that guarantee both safety and task completion. We validate our framework through simulation studies and real-world experiments on a quadrupedal robot, demonstrating its effectiveness in hybrid mode planning and safety-critical applications.