Real-Time Control-Constrained DDP for Underactuated Balancing of Legged Robots

AI summary

Overview

Research area: Robotics — optimal control and model predictive control (MPC) for legged locomotion, specifically control-constrained Differential Dynamic Programming (DDP) applied to underactuated balancing.

Technical level: Advanced. The paper assumes familiarity with nonlinear optimal control, Bellman's principle of optimality, DDP/iLQR backward and forward passes, KKT systems, and active-set methods.

One-sentence scope: The paper proposes ABC-DDP, a real-time control-constrained DDP framework built on an Accelerated Projected Gradient (APG) solver and a virtual-constraint multiple-shooting scheme, and uses it to achieve static two-leg standing plus dynamic locomotion of a quadruped within a single finite-horizon MPC framework in simulation.

What This Paper Is About

Classical DDP handles nonlinear dynamics constraints well but does not explicitly enforce control limits such as box constraints or friction cone constraints, which are exactly the constraints legged robots need. Existing fixes are fragmented: Box-DDP exploits box structure but not general linear constraints, Constrained DDP (CDDP) uses an indirect active-set treatment that can require KKT inversions at every sub-iteration, and FDDP/Box-FDDP can handle dynamic infeasibility and box limits but only in an alternating, not unified, manner. The paper's goal is a single real-time MPC framework that handles linear control constraints, multiple shooting, and strong underactuation together, and demonstrates it on a quadruped balancing on a support line (two-leg standing) as well as walking and running.

Key Contributions

1. An APG-based control-constrained DDP (ABC-DDP): By incorporating the Nonmonotone Accelerated Projected Gradient (APG) method, the framework handles linear control constraints with exact active-set identification without repeated KKT inversions, extending beyond the box-only setting of Box-DDP and the indirect active-set treatment of CDDP.
2. A unified control-constrained multiple-shooting DDP framework: The authors introduce a novel virtual constraint formulation so that control constraints and a multiple-shooting scheme are treated simultaneously. The paper states that no existing method considers both at once — Box-DDP and FDDP address box-constrained mode and unconstrained multiple-shooting mode alternately within Box-FDDP.
3. Static two-leg standing of a quadruped within real-time finite-horizon MPC: To the authors' knowledge this is the first such demonstration. The contact-implicit MPC formulation of prior work (solved with Box-FDDP) produced two-leg balancing but not static standing with fixed foot contacts, because the optimizer simultaneously planned separating, clamping, and sliding contact phases.
4. Diverse dynamic motions inside one framework: slow catwalk, upright bipedal walking, and high-speed running at 6.0 m/s are all generated within a single MPC framework in real-time simulation.

Main Findings

• Static two-leg standing works under disturbances: Simulation results demonstrate static two-leg standing under external disturbances. The authors note that two-leg standing on a point-foot quadruped corresponds to balancing on a support line (not a polygon), analogous to a human balancing on tiptoes without taking steps, and generally requires exploiting rotational dynamics including precession and nutation effects.
• Short horizons suffice despite strong underactuation: The paper reports that underactuated conditions can be sustained with a short finite-horizon prediction, contrary to the common belief that long-horizon prediction is required (prior single-step QP work approximated the constrained cost-to-go with its unconstrained counterpart).
• One framework covers four motion types: The four representative motions are static two-leg standing, slow catwalk, upright walking, and high-speed running at 6.0 m/s.
• APG supplies the constrained solution and the active set: Solving the DDP subproblem with the feedforward term set to a zero state deviation lets APG identify the active set naturally as a byproduct of optimization; the KKT system is then solved using that active set.
• Convergence properties: With Nesterov-type momentum, APG achieves a convergence rate of O(1/k²) in locally convex regions. The nonmonotone acceptance criterion (controlled by η ∈ [0,1)) improves convergence in ill-conditioned settings and ensures convergence even for nonconvex problems, so the DDP subproblem can be solved within the feasible set even when the problem is not strictly positive definite.
• Feasibility from bad initializations: The FDDP-based state gap lets the optimizer start from a dynamically infeasible rollout that stays bounded around the desired trajectory instead of diverging, which the authors note would be inevitable under single shooting with prolonged underactuation.
• Virtual constraint and boundary shrinkage: The backtracked feedforward step is reinterpreted as a hyperplane normal to the gradient, along which the feedback component slides; the original constraint set is additionally contracted by a factor β ∈ (0,1] to keep the feedback step near the physically meaningful boundary.
• Robot model and parameters used: A Single Rigid Body Dynamics (SRBD) model is used, with body mass 37.5 kg, body length 0.6448 m, body width 0.119 m, inertia Ixx = 0.7, Iyy = 2.8, Izz = 3.3 kg·m², scap length 0.1377 m, hip length 0.35 m, knee length 0.4136 m, and friction coefficient μ = 0.5. Control inputs are the ground reaction forces (GRFs) per leg, constrained by a linear friction pyramid plus box limits, with swing-phase legs set to zero vertical force bounds.
• Swing times differ per gait: Reported swing times T_sw are not applicable (N/A) for two-leg standing, 0.6 for catwalk, 0.2 for upright walking, and 0.08 for running.

Methodology in Plain English

The authors formulate locomotion as an MPC problem: minimize a sum of weighted quadratic state-error and control costs over a horizon, subject to the discretized single-rigid-body dynamics and to linear friction-pyramid and box constraints on the ground reaction forces.

Standard DDP solves the dynamics-constrained problem efficiently with second-order approximations, but ignores the force limits. The authors instead solve the force-limited part of each DDP subproblem with an accelerated projected gradient method — a first-order method that only needs gradient steps plus an orthogonal projection, which for boxes and friction cones has a closed-form solution. Because this is cheap, it can be run inside the backward pass of every time step. The active constraints found by APG are then used to build and solve a small KKT system that yields the feedback gain, avoiding repeated KKT inversions for active-set exploration.

To cope with underactuated, hard-to-initialize problems, the authors adopt the FDDP idea of a "state gap" — treating the integrated next state and the stored next state as independent and measuring their discrepancy — which gives multiple-shooting behavior and lets the optimizer start from a dynamically infeasible rollout that gradually becomes feasible as a step-size parameter α shrinks the gap. A PD-like Newton–Euler control law with projection onto the constraint set generates that initial rollout.

The key new ingredient is the virtual constraint: when backtracking is active, the scaled feedforward step is reinterpreted as a hyperplane constraint, so the feedback correction slides along that hyperplane instead of re-entering the constraint interior arbitrarily. The admissible region is also contracted by a factor β, and a second-order expected-improvement test with adjustable parameters b1 and b2 decides whether to accept a backtracking step, allowing moderate cost increases early on so iterations can prioritize feasibility. The whole loop — gap computation, backward pass, KKT solve, forward pass, line search — is summarized in Algorithm 2.

Why This Matters

Impact on research: The paper argues that constrained DDP methods have been fragmented, and that acceleration-based first-order methods (APG) had received limited attention in the control community. Showing that a projection method can serve as the constrained solver inside DDP — and that control constraints plus multiple shooting can live in one framework — opens a route to principled handling of general linear constraints in real-time MPC for legged systems. It also challenges the assumption that long-horizon prediction is necessary for underactuated balancing.

Real-world applications (implied by the demonstrated behaviors):

• Quadruped robots that must stand still on a narrow support line (for example, narrow-foothold or ledge situations) rather than continuously stepping.
• Legged platforms performing multiple gait types — slow catwalk, upright walking, and high-speed running at 6.0 m/s — without switching control stacks.
• Robots operating under strong underactuation where dynamic feasibility is hard to initialize, benefiting from feasibility-driven initialization.
• Balancing tasks where friction limits and force limits must be respected exactly, since the friction pyramid and box constraints are enforced explicitly.

Industry relevance: The work is published through IEEE Robotics and Automation Letters (DOI 10.1109/LRA.2026.3723262), received March 31, 2026 and accepted July 20, 2026, and was funded by the Materials and Parts Technology Development Program (RS-2024-00508191) of MOTIE, Korea, tied to unmanned autonomous operation technology with field-use visualization and 6-axis rotational angle sensors. One co-author is affiliated with AIDIN ROBOTICS Inc., indicating direct commercial interest in legged robot control. Real-time capability is central: the framework is designed for control-limited MPC running online, and a single unified controller reduces the engineering burden of maintaining separate constrained and unconstrained solvers.

Future Directions

• Hardware validation. The paper reports simulation results only; demonstrating two-leg standing, catwalk, upright walking, and 6.0 m/s running on physical hardware, and reporting computation times, is the natural next step.
• Extending beyond simple constraint sets. The authors explicitly note that APG is well suited to constraints with closed-form projections (boxes, friction cones) but may be less effective for constraints whose projections are difficult to compute, such as nonlinear dynamics constraints — leaving open how far the approach generalizes.
• Longer horizons and more dynamic motion. The paper notes that integration errors can accumulate and lead to divergence for long horizons or highly dynamic trajectories with large fluctuations, so scaling the approach under those conditions remains an open question.
• Comparison against alternative constrained DDP variants. The paper motivates ABC-DDP against Box-DDP, CDDP, and Box-FDDP conceptually, but the provided content does not report head-to-head benchmarks, convergence tables, or per-iteration timing comparisons — evaluating those would clarify the practical gains of exact active-set identification versus the cheaper box-specific or heuristic alternatives.

Target Audience

Robotics researchers and graduate students working on optimal control, model predictive control, and legged locomotion, especially those already familiar with DDP/iLQR and active-set methods. It also suits control engineers at legged-robot companies interested in a single real-time MPC framework that spans standing, walking, and running, and practitioners looking for alternatives to QP-based MPC formulations that require approximated cost-to-go terms. Readers without a background in nonlinear optimal control will find the constraint-handling and multiple-shooting machinery difficult without additional study.

Authors’ abstract

This paper presents a real-time control-constrained Differential Dynamic Programming (DDP) framework for underactuated legged robots. To address the limitation of classical DDP in handling control constraints, we propose an Accelerated Projected Gradient (APG)-based control-constrained DDP (ABC-DDP), which efficiently computes constrained solutions and identifies active sets without repeated Karush-Kuhn-Tucker (KKT) inversions. A virtual constraint is introduced to integrate control constraints within a feasibility-driven multiple-shooting framework, enabling stable optimization even from dynamically infeasible initializations. The proposed method supports real-time model predictive control (MPC) with short horizons under strong underactuation. Simulation results demonstrate static two-leg standing under external disturbances, along with diverse dynamic motions including slow catwalk, upright walking, and high-speed running within a unified MPC framework. To the best of our knowledge, this is the first demonstration of static two-leg standing of a quadruped robot achieved using real-time finite-horizon MPC.