Certified Gradient-Based Contact-Rich Manipulation via Smoothing-Error Reachable Tubes

AI summary

Overview

• Research area: Robotics — contact-rich manipulation, differentiable simulation, trajectory optimization, and set-valued robust control (reachability analysis).
• Technical level: Advanced. The paper leans on conic programming, KKT complementarity relaxation, implicit differentiation, and robust reachable-set theory.
• Scope (one sentence): The paper builds a differentiable contact simulator that smooths both contact dynamics and contact geometry, analytically bounds the error that smoothing introduces relative to the true nonsmooth hybrid dynamics, and uses those bounds to synthesize affine feedback policies that are certified to satisfy constraints on the original nonsmooth system.

What This Paper Is About

Gradient-based manipulation planners work well when the underlying physics is differentiable, but real contact is nonsmooth: gradients vanish when objects are not yet touching and jump discontinuously across contact modes. Smoothing the physics restores usable gradients, but then the model no longer matches reality, so controllers that look good in the smoothed model can fail on the true system. This paper closes that gap by measuring the smoothing-induced error exactly enough to bound it, propagating it forward as reachable tubes, and optimizing feedback policies that remain safe and reach their goals under the original nonsmooth dynamics.

Key Contributions

1. A modular differentiable simulator with two kinds of smoothing. Contact dynamics and contact geometry are smoothed separately within differentiable convex conic programs, yielding accurate planning gradients without differentiating through the entire computation graph.
2. Tight analytical bounds on smoothing error. The paper derives a set-valued, state- and control-dependent bound on the deviation between the smoothed dynamics and the nonsmooth hybrid dynamics, built from gradients the simulator provides (including the sensitivity of the solution to the smoothing parameter κ).
3. A robust policy synthesis method. Gradients from the smoothed dynamics drive efficient optimization, while tube-valued predictions of the nonsmooth hybrid dynamics under feedback control enforce robust constraint satisfaction.
4. Demonstration on contact-rich tasks in simulation and on hardware, including planar pushing (2DOF), bimanual non-prehensile manipulation (9DOF), and dexterous in-hand reorientation (22DOF), with reduced constraint violation rate relative to baselines.

Main Findings

• The smoothing error can be bounded exactly in form. Theorem 1 states that under the assumption that J_i P^{-1} J_j^⊤ = 0 for all i ≠ j, the nonsmooth dynamics equal the smoothed dynamics minus P(x)^{-1} Σ_i J_i(x)^⊤ (∂λ_{κ,i}/∂κ) κ w_i, where w_i ∈ [1, 2]. The bound is described as tight: for every choice of w_i in {1, 2}, there exist (x, u) pairs satisfying the expression.
• The bound is intrinsically one-sided. Because w_i is one-sided rather than centered around zero, the discrepancy reflects the unilateral nature of contact — contact can only push, not pull.
• The bound is differentiable and compactly written. Equation (7) expresses the next state as f_κ(x, u) + E_κ(x, u) w with w ∈ [1, 2]^{n_c}, where column i of E_κ is −P(x)^{-1} J_i(x)^⊤ (∂λ_{κ,i}/∂κ) κ, and both f_κ and E_κ are continuously differentiable.
• Recenterning the nominal dynamics would destroy the gradients. The paper's 1D pusher example shows that choosing f_κ(x, u) + 1.5 E_κ(x, u) as the nominal model (to remove bias) yields zero or even negative ∂x_o^+/∂u gradients over parts of the input domain, making them non-informative for action refinement.
• Diagonal dominance, not strict orthogonality, appears sufficient in practice. Remark 1 plots terms in equation (6) against the diagonal dominance factor of J P^{-1} J^⊤ for the bimanual planar bucket manipulation example. Equation (6) appears to hold once the diagonal dominance factor exceeds approximately 0.5; across the examples in the paper the factor ranges from 1.3 to 2.1, exceeding that 0.5 cutoff.
• Closed-loop behavior under feedback is analytically representable. Theorem 2 shows that the set of all possible closed-loop responses satisfies x = z + Φ^x w and u = v + Φ^u w with w ∈ [1, 2]^{N·n_c}, and that an affine feedback policy u = v + Φ^u(Φ^x)^{-1}(x − z) = v + K(x − z) realizes such a response.
• Robustness comes from a closed-form worst case. Because w lies in the set {w | ||w − w_c||_p ≤ w_r} with w_c = 1.5·1, w_r = 0.5, and p = ∞, the worst-case value of a^⊤w is a^⊤w_c + w_r ||a||_q (q the dual norm exponent with 1/p + 1/q = 1), giving the row-wise robustified constraints in the optimization.
• Certification on the nonsmooth system. The abstract reports certified constraint satisfaction with lower safety violations and smaller goal errors than baseline approaches; the contributions state that the method reduces constraint violation rate relative to baselines.
• Comparable solve time to the contact-trust-region baseline. Section V-C states the method obtains similar solve times to trajectory optimization with contact trust region while reducing constraint violations and certifying constraint satisfaction on the nonsmooth hybrid system. Specific numerical solve times are not reported in the provided content.
• Quantitative results tables are not included in the provided content. No numeric success rates, violation percentages, or goal-error magnitudes appear in the text supplied; only the qualitative comparisons above are reported.

Methodology in Plain English

The researchers start from a quasistatic model of contact — one that ignores velocities because manipulation transients are brief — and write the next state as the solution of a convex conic program built from the mass matrix, contact Jacobians, and signed distances. The true version enforces complementarity, meaning contact force and signed distance cannot both be positive. They relax that condition with a parameter κ, which turns the problem into a smooth, barrier-penalized convex program. The relaxed program has well-behaved, informative gradients, and because it is a convex program they can differentiate its solution implicitly with respect to state, control, and κ itself, without backpropagating through an entire computation graph.

They then derive a formula showing exactly how far the smoothed dynamics drift from the true nonsmooth dynamics. That drift turns out to be a matrix multiplying an unknown vector w confined to the box [1, 2] — a set-valued discrepancy rather than a single number. Notably, the paper also smooths contact geometry, formulating contact-point queries between a sphere and a polyhedron as a relaxed quadratic program, though it explicitly defers bounding the error from geometry smoothing to future work, arguing that this error is a local constant offset rather than the accumulating "force-at-a-distance" error from dynamics smoothing.

The planning problem optimizes a nominal state and control trajectory plus a time-varying affine feedback policy. The feedback policy is parameterized by lower-triangular block matrices Φ^x and Φ^u, which propagate the unknown w through the closed loop. Since the resulting state and control are affine in w, and w lives in a box, the worst case over the box has a closed form. Substituting that worst case into the state, control, and terminal constraints turns them into robust constraints that hold for every possible realization of the smoothing error. A sequential convex programming loop linearizes the smoothed dynamics and the discrepancy around the current trajectory, solves the resulting convex subproblem with a step-size trust region, and repeats. After convergence, the gains K are recovered as Φ^u(Φ^x)^{-1}.

The baseline is a prior contact-trust-region method related to the iterative linear quadratic regulator, which instead constrains the linearized contact force to stay inside the friction cone. The paper's example figures contrast a linearization of the smoothed dynamics against the same linearization augmented with the deviation bounds, and show nominal versus rollouts on the nonsmooth dynamics with and without predicted tubes.

Why This Matters

Impact on research. The paper offers a bridge between two communities that usually work separately: differentiable simulation, which supplies the gradients, and set-valued reachability, which supplies the guarantees. It replaces a heuristic safety mechanism (a contact trust region with no formal characterization of allowable deviation) with an analytically derived, differentiable error bound. It also produces feedback policies rather than open-loop trajectories, which reduces the need for continual replanning in a receding-horizon loop.

Real-world applications (grounded in the evaluated tasks):

• Warehousing and logistics, where planar pushing of unactuated objects is a common primitive.
• Dexterous robotic hands performing in-hand reorientation and regrasping of objects.
• Bimanual, non-prehensile manipulation such as moving a bucket-like container using two arms without grasping it.
• Any contact-rich assembly or tool-use setting where joint limits, torque limits, non-collision, and workspace bounds must be respected at execution time.

Industry relevance. Manufacturers deploying manipulation policies care about a certificate, not just a lower average error — a policy that is proven to keep the true system inside its constraints is far easier to validate and sign off than one that only performed well in the smoothed simulator. The paper reports the method runs at solve times comparable to the contact-trust-region baseline, which matters for practical deployment, and it targets deployment on hardware rather than simulation alone.

Future Directions

• Bounding geometry-smoothing error. The paper characterizes only dynamics smoothing; it explicitly defers analysis of how contact-geometry smoothing perturbs the contact Jacobian and signed distance, noting this error is less severe and manifests as a local constant offset.
• Accounting for linearization error. The formulation does not explicitly account for the model mismatch introduced by approximating the nonlinear tracking-error dynamics as a linear time-varying system. The authors suggest this could be handled with linearization error bounds computed via interval arithmetic or statistical methods, inflating the tubes to maintain guarantees.
• Relaxing the orthogonality assumption. Theorem 1 assumes J_i P^{-1} J_j^⊤ = 0 across distinct contacts. Remark 1 suggests diagonal dominance above roughly 0.5 is sufficient in practice over the diagonal-dominance factors of 1.3 to 2.1 observed in the examples, but a formal treatment of the non-orthogonal case remains open.
• Scaling and global search. The related-work discussion notes that gradient-based trajectory optimization only guarantees convergence to local minima, and that global methods such as graph of convex sets, polynomial optimization, mixed-integer programming, and rapidly exploring random trees remain restricted to low-dimensional systems and vulnerable to combinatorial growth in the number of contact modes.

Target Audience

Robotics researchers and graduate students working on manipulation, differentiable simulation, trajectory optimization, or reachability-based safety. Practitioners building contact-rich manipulation systems — dexterous hands, bimanual platforms, or pushing-based logistics robots — who need formal constraint-satisfaction guarantees rather than empirical reliability will find the certification machinery most useful. Readers without a background in convex conic programming, complementarity conditions, or reachable-set robustness should expect a steep curve; the paper is written at an advanced level and relies on implicit differentiation formulas relegated to its appendices.

Authors’ abstract

Gradient-based methods can efficiently optimize controllers by leveraging differentiable simulation and physical priors. However, contact-rich manipulation remains challenging because hybrid contact dynamics often produce discontinuous or vanishing gradients. Although smoothing the dynamics can restore informative gradients, the resulting model mismatch can cause controller failures when deployed on real systems. We address this trade-off by planning with smoothed dynamics while explicitly quantifying and compensating for the induced error, providing formal guarantees on safety and task completion under the original nonsmooth dynamics. Our approach applies smoothing to both contact dynamics and contact geometry within a differentiable simulator based on convex optimization, allowing us to characterize the deviation from the nonsmooth dynamics as a set-valued discrepancy. We incorporate this discrepancy into the optimization of time-varying affine feedback policies through analytical reachable sets, enabling robust constraint satisfaction for the closed-loop hybrid system while relying solely on the informative gradients of the smoothed model. By bridging differentiable simulation with set-valued robust control, our method produces affine feedback policies that respect the unilateral nature of contact. We evaluate our method on several contact-rich tasks, including planar pushing, object rotation, and in-hand dexterous manipulation, achieving certified constraint satisfaction with lower safety violations and smaller goal errors than baseline approaches.