Learning Agile Quadrotor Flight in the Real World

AI summary

Overview

• Research area: Robotics — learning-based control for agile quadrotor flight, real-world (on-robot) adaptation, differentiable simulation, and sim-to-real-free policy learning.
• Technical level: Advanced. The paper assumes familiarity with rigid-body quadrotor dynamics on SO(3), collective-thrust/body-rate (CTBR) control interfaces, backpropagation through time, differentiable simulation, and Lie-group integration.
• Scope: The paper presents a self-adaptive flight framework that replaces offline system identification and sim-to-real transfer with online residual dynamics learning plus in-flight policy optimization, and validates it on a physical quadrotor performing agile maneuvers, hardware-degradation tests, and a wind-disturbed inspection mission.

What This Paper Is About

Most learning-based agile quadrotor controllers are trained in simulation with domain randomization and then frozen, so they cannot cope with non-stationary real-world conditions such as hardware wear, battery depletion, payload changes, or aerodynamic disturbances, and they must fly with conservative safety margins. This paper asks whether a quadrotor can instead improve its own low-level control directly in the real world — safely and with very little flight data — starting from a conservative base policy and pushing toward the agility ceiling imposed by its CTBR command interface.

Key Contributions

1. Adaptive Temporal Scaling (ATS) — a closed-loop mechanism that treats the reference trajectory's time-dilation factor α as an optimizable parameter, jointly optimized with the policy using analytical gradients from closed-loop sensitivity, so the system automatically compresses its time scale when tracking is precise and relaxes it when disturbances appear. Safety is enforced via a Softplus barrier on tracking error relative to a threshold E_th.
2. Online residual learning on a hybrid dynamics model — a neural residual term (translational acceleration residual a_res and body-rate residual ω_res) augments a simple nominal CTBR rigid-body model, trained by minimizing one-step integrated state-prediction error from a sliding-window replay buffer, with a spectral-norm regularizer to bound the residual's Lipschitz constant.
3. Real-World Anchored Short-Horizon Backpropagation Through Time (RASH-BPTT) — in-flight policy updates that initialize every differentiable rollout from the most recent real-world state estimate (anchoring), rather than random resets, over a short horizon to prevent compounding model error and model exploitation.
4. A self-adaptive framework that removes the need for precise system identification or offline sim-to-real transfer, demonstrated on real hardware executing near CTBR saturation, adapting to hardware damage and payload, and reducing inspection mission time under unknown wind.

Main Findings

• Speed evolution on the linear shuttle: the system evolves a conservative base policy from a peak speed of 2.0 m/s to 7.3 m/s within approximately 100 s of flight time.
• Speed evolution on the Figure-8: speed increases from 2.2 m/s to 7.2 m/s over the same adaptation horizon.
• The figure caption describes this as "tripling its speed within 100 s of flight time across three tasks," comparing Iter 0 (conservative) with Iter 12 (evolved agile behavior), with trajectories composited from frames sampled at uniform 0.5 s intervals (sparser segments indicate higher velocity).
• Operation at the command-interface saturation limit: commanded body rate ω_cmd persistently saturates at ‖ω_cmd‖∞ = 6 rad/s during later stages of adaptation, on both the Figure-8 and Linear Shuttle, as shown by raw commands and their 2 s sliding-window mean.
• Comparison to zero-delay offline lower bounds (Table I): Figure-8 — rotor-thrust bound 2.02 s, CTBR bound 2.24 s, Ours 2.60 s; Shuttle — rotor-thrust bound 1.42 s, CTBR bound 1.49 s, Ours 2.30 s. The rotor-thrust bound is marked as not reachable through the CTBR interface because the optimized trajectory requires body rates exceeding 28 rad/s.
• Figure-8 gap analysis: the 2.60 s result approaches the CTBR bound of 2.24 s, indicating operation near the command-interface ceiling; the larger shuttle gap (2.30 s vs 1.49 s) is attributed to that trajectory's stronger sensitivity to actuation delays, rapid acceleration reversals, and aerodynamic effects absent from the zero-delay reference.
• Robustness to hardware variations: under added mass (60 g payload), propeller damage (clipped tips), and combined conditions, achievable maximum speeds naturally decreased, but the framework identified each configuration's new feasible operating envelope and operated at the boundary of CTBR body-rate saturation. After just a single update iteration of the residual model, tracking error dropped precipitously, letting ATS decrease α and increase agility.
• Platform degradation detail: the nominal quadrotor is 192 g; the 60 g payload is a 31% mass increase that alters inertial properties and reduces the thrust-to-weight ratio.
• Inspection mission under unknown wind: after only 2 min of real-world training, mission completion time was reduced by 42% (from 12 s to 7 s) while maintaining low tracking error, comparing Policy V0 to Policy V10.
• Wind-disturbance comparison: the Baseline fails to accelerate because of unmodeled wind drag and remains stuck at 4.7 m/s; Ours first suppresses tracking error with the residual network (V0 to V1, RMSE drops to 0.06 m) to create a safety margin, then ATS scales agility up to a top speed of 6.8 m/s (V10) while keeping tracking error within threshold.
• Ablation (simulation, Fig. 7): all compared methods — Baseline (nominal), LOTF [17] (acceleration residual only), component ablations (Anchor/Residual Only), and Ours (full system) — maintain bounded tracking error below E_th and reduce lap time when augmented with ATS, but Ours achieves the shortest lap time, indicating anchored initialization plus full hybrid residual modeling is critical for pushing the agility envelope.
• Contrast with prior real-world adaptation: the authors state that prior differentiable-simulation adaptation work prioritizes disturbance rejection over pushing the agility envelope, yielding conservative performance typically around 1 m/s to 2 m/s.
• Compute and loop rates: motion capture provides state estimates at 100 Hz; an off-board workstation runs the online learning loop and sends commands to the onboard flight controller at 50 Hz; each adaptation step (100 epochs: ~3 s residual + ~6 s policy) completes in ~9 s on an NVIDIA RTX 4090 GPU using JAX with JIT and vmap; the 50 Hz command loop needs only a forward pass through a lightweight MLP (~100 μs), decoupling policy execution from learning.
• Not reported: the truncated content does not include the simulation details beyond the ablation setup, nor does it report convergence curves, failure rates, or a full list of quantitative ablation numbers.

Methodology in Plain English

The system runs as a continuous loop between flying the real drone and optimizing in simulation.

Step 1 — Start simple. A base control policy is pre-trained with standard backpropagation through time on a plain nominal quadrotor model. Because differentiable simulation gives first-order gradients instead of noisy zeroth-order reinforcement learning estimates, this pre-training converges within seconds.

Step 2 — Learn what the model gets wrong. Once flying, the drone collects state-action-next-state transitions from its state estimator into a sliding-window buffer. A neural residual network takes position, velocity, the vectorized rotation matrix, and the control command as input, and outputs corrections: an extra translational acceleration and an extra body-rate offset that are added to the nominal physics. Crucially, training minimizes the error of the integrated next state (not noisy acceleration targets), using a differentiable fourth-order Runge-Kutta integrator — implemented as a Lie-group integrator that updates the rotation on SO(3) with the exponential map — so the loss respects rotational geometry and position/velocity errors can supervise attitude residuals. A layer-wise spectral-norm penalty keeps the residual smooth enough for stable long-horizon optimization.

Step 3 — Optimize the policy in flight. The policy is a small multilayer perceptron (two hidden layers of 256 units) that maps current state, reference trajectory state, and past actions to a four-dimensional CTBR command. At each real-world time step, the method anchors a differentiable rollout at the drone's current measured state rather than a random reset, unrolls the learned hybrid dynamics for a short horizon H, and computes policy gradients by backpropagation through time through the RK4 integrator and residual network using JAX autodiff, with JIT and vmap for parallel speed. Short horizons plus anchoring are what keep gradients grounded in reality instead of drifting into model-exploiting fantasies.

Step 4 — Stretch time itself. Instead of heuristically speeding up the reference trajectory, the method parameterizes the trajectory as a piecewise polynomial and introduces a scalar time-dilation factor α, so the reference at trajectory time τ is evaluated at τ/α. Since the physical system is not differentiable, the hybrid model serves as a differentiable proxy for "counterfactual inference": the method linearizes the hybrid dynamics along the real rollout to compute how the closed-loop state and tracking error would change with α. It then maximizes a composite objective balancing speed (λ_speed · α, which rewards small α) against a Softplus barrier on tracking error above threshold E_th, updating α by projected gradient descent within bounds. In the safe regime the barrier gradient vanishes and α is driven down for speed; in the unsafe regime the barrier dominates and α increases to restore feasibility — producing an equilibrium near the safety boundary.

Testing. Everything is validated on a physical quadrotor based on the Agilicious platform, with motion capture at 100 Hz and off-board control commands at 50 Hz, plus simulation for the ablation study.

Why This Matters

Impact on research. The paper reframes real-world adaptation as more than an error-compensation trick: it argues that online learning is a practical mechanism for sustained performance improvement in aggressive flight regimes, not merely a way to close a sim-to-real gap. It shows that a controller can approach the saturation limit of its own command interface without offline system identification, and it introduces an anchored short-horizon differentiable-simulation formulation that addresses the long-standing model-exploitation problem in differentiable simulation for neural policies.

Real-world applications:

• Multi-point infrastructure inspection under wind or aerodynamic disturbance, where mission time directly determines cost and battery feasibility — the paper's own demonstration.
• Autonomous drone racing and time-trial flight, where operating near the command ceiling determines competitive outcomes.
• Payload delivery and logistics, where payload mass varies between trips and changes the vehicle's dynamics mid-deployment.
• Flight after damage or wear — clipped propellers, degraded motors, aging batteries — where a fixed policy would simply underperform or become unsafe, but residual learning can re-identify the new feasible envelope.

Industry relevance. The framework eliminates expensive offline system identification and repeated retraining cycles, which is attractive for fleets where each airframe differs slightly and degrades over time. The reported compute profile matters commercially: the control loop is a ~100 μs forward pass through a small MLP, and learning is decoupled onto an off-board GPU, so this maps naturally onto a ground-station-assisted architecture with a lightweight onboard controller — while pointing toward fully onboard deployment as embedded compute improves.

Future Directions

• Onboard deployment. The current learning loop runs on an off-board workstation (RTX 4090) sending commands to the flight controller; moving residual learning and RASH-BPTT fully onboard is the obvious next step and remains an open engineering question.
• Perception beyond motion capture. State estimates come from a 100 Hz motion-capture system, so extending the framework to onboard visual-inertial state estimation — with its noise and latency — is untested and would be needed for field use.
• Characterizing the gap to offline optima. Ours reaches 2.60 s versus a CTBR bound of 2.24 s on the Figure-8 and 2.30 s versus 1.49 s on the shuttle; the causes named are actuation delays, rapid acceleration reversals, and aerodynamic effects, so closing that gap (e.g., explicit delay modeling) is a concrete open problem.
• Generality of the residual model and ATS. The residual network is trained on data from the current deployment and is demonstrated on specific trajectories and three hardware conditions; how well it transfers across trajectories, environments, or vehicle classes — and whether the α equilibrium stays well-behaved over long missions with shifting wind — is not established.

Target Audience

Robotics researchers and graduate students working on learning-based control, differentiable simulation, and sim-to-real transfer, particularly those focused on aerial robotics and aggressive flight. It is also relevant to practicing drone engineers evaluating online adaptation for fleets subject to payload variation and hardware degradation, and to reinforcement-learning researchers interested in first-order gradient methods as an alternative to sample-hungry zeroth-order policy optimization. Readers without a background in Lie-group dynamics, BPTT, or differentiable simulation will need to consult the cited background work to follow the methodology.

Authors’ abstract

Learning-based controllers have achieved impressive performance in agile quadrotor flight but typically rely on massive training in simulation, necessitating accurate system identification for effective Sim2Real transfer. However, even with precise modeling, fixed policies remain susceptible to out-of-distribution scenarios, ranging from external aerodynamic disturbances to internal hardware degradation. To ensure safety under these evolving uncertainties, such controllers are forced to operate with conservative safety margins, inherently constraining their agility outside of controlled settings. While online adaptation offers a potential remedy, safely exploring physical limits remains a critical bottleneck due to data scarcity and safety risks. To bridge this gap, we propose a self-adaptive framework that eliminates the need for precise system identification or offline Sim2Real transfer. We introduce Adaptive Temporal Scaling (ATS) to actively explore platform physical limits, and employ online residual learning to augment a simple nominal model. {Based on the learned hybrid model, we further propose Real-world Anchored Short-horizon Backpropagation Through Time (RASH-BPTT) to achieve efficient and robust in-flight policy updates. Extensive experiments demonstrate that our quadrotor reliably executes agile maneuvers near actuator saturation limits. The system evolves a conservative base policy with a peak speed of 1.9 m/s to 7.3 m/s within approximately 100 seconds of flight time. These findings underscore that real-world adaptation serves not merely to compensate for modeling errors, but as a practical mechanism for sustained performance improvement in aggressive flight regimes.