DriveZero: End-to-End Driving Beyond Human Demonstrations

AI summary

Overview

Research area: End-to-end autonomous driving, specifically closed-loop reinforcement learning, vision foundation model distillation, and camera-only trajectory planning.

Technical level: Advanced. The paper assumes familiarity with PPO, privileged policy distillation, vision transformers, and the nuPlan / NAVSIM / HUGSIM evaluation suites.

Scope: This technical report from Xiaomi EV's AD & Robotics, L3 Team describes a three-stage system — DriveRL (an RL-trained privileged teacher), DriveVFM (a perception backbone distilled from frozen vision foundation models), and DriveZero (a camera-only student planner) — that learns driving behavior without imitating human trajectories.

What This Paper Is About

Most end-to-end driving systems are trained by imitating recorded human driving logs, which limits the learned policy to the quality and coverage of those logs; each logged scene contains only one realized future, safety-critical recovery maneuvers are rare, and states induced by the learned policy are absent from the offline data. The paper asks whether driving behavior can instead be learned from closed-loop reinforcement learning and then distilled into a deployable camera-only planner, so that the system is not constrained by human demonstrations. The goal is a single end-to-end system whose supervision comes from reinforcement-learned behavior rather than from logged human trajectories.

Key Contributions

1. DriveRL, a mixed-agent closed-loop RL framework that converts real nuPlan logs into interactive worlds where log replay, rule-based behaviors (IDM, front-vehicle braking), and learned policies coexist. It trains a 5.70M-parameter privileged teacher from scratch with PPO, starting from random initialization with no imitation pretraining, and adds a value-guided test-time action search that reranks policy-supported action candidates.

2. DriveVFM, a perception backbone that consolidates multiple frozen vision foundation models — DINOv3, SigLIP2, SAM, and Depth Anything V2 — into a single driving backbone from raw images alone, requiring no detection, segmentation, lane, or depth annotations. It uses PHI Standardization (PHI-S) to balance the heterogeneous feature statistics of the teachers and a QK-Clip-inspired intervention to stabilize attention logits.

3. DriveZero, a camera-only end-to-end planner that initializes its encoder from DriveVFM and learns by winner-takes-all distillation against trajectories rolled out by the frozen DriveRL teacher, with a separate scoring head that predicts six PDM score components. Its training signal comes from reinforcement-learned behavior, not human demonstrations.

4. Goal-conditioned augmentation, which exploits the fact that the RL teacher is goal-conditioned: the same logged scene can be queried under augmented driving intents and navigation commands, yielding diverse, goal-consistent supervision that logged data cannot provide.

Main Findings

• DriveRL exceeds the demonstrations that seeded it. DriveRL achieves an unweighted mean score of 93.01 across Val14, Test14-hard, and Test14-random in both non-reactive (NR) and reactive (R) modes, exceeding the Log-Replay expert in every one of those settings. It also exceeds CaRL and GigaFlow, the two prior methods trained without human data, on every setting they report.

• Test-time search improves a fixed checkpoint without retraining. Value-guided test-time action search (DriveRL-TTS) raises the mean from 93.01 to 93.57, improving five of the six settings while remaining essentially unchanged on Test14-hard R (89.15 vs. 89.18). At the full budget of 64 candidates, the largest gains are on non-reactive splits: Test14-hard NR improves by 1.16 points and Test14-random NR by 1.43 points.

• DriveZero matches the human driver on NAVSIMv1 without human trajectory supervision. DriveZero reaches 94.8 PDMS on navtest, the same value reported for the human driver (94.8) in the same table. The abstract and the results table describe the scaled variant, DriveZero-Scale, as reaching 95.3 PDMS; the introduction describes the 94.8 figure as "on par with the human driver" and attributes the 95.3 figure to scaling with simulation data.

• Scaling with simulated data sets state of the art across three benchmarks. DriveZero-Scale reaches 95.3 PDMS on NAVSIMv1 navtest, 57.1 EPDMS on NAVSIMv2 navhard, and 46.6 HD-Score zero-shot on the closed-loop HUGSIM benchmark. Zero-shot means no HUGSIM-specific finetuning is used.

• The RL teacher itself is a strong planner when given ground-truth symbolic inputs. On the NAVSIMv1 navtest table, DriveRL with GT inputs scores 95.8 PDMS, the highest PDMS in that table.

• Goal augmentation beats human-trajectory supervision in ablations. The paper reports that teacher supervision with goal augmentation surpasses human-trajectory supervision, and that an additive study shows each teacher foundation model contributes cumulatively to DriveVFM. Numerical values for these ablations are not reported in the provided content.

• Self-play helps deployment more than the benchmark. Policy-controlled background vehicles expose DriveRL to more natural interactions, but the benefit on nuPlan is small; the authors attribute this to nuPlan's closed-loop evaluation driving background vehicles by IDM. Self-play is the configuration used for real-world deployment. Configuration and results are referenced to Tables A6 and A7, whose contents are not in the provided text.

Methodology in Plain English

The authors deliberately split driving into two problems and solve each in the setting it suits best, then reunite them.

Learning how to act (DriveRL). They build interactive worlds out of real nuPlan logs. Every other road user in a scene gets its own behavior provider through a common physical state–action interface, so replayed traffic, rule-based drivers, and learned policies can share one scene. The ego policy is a compact network (5.7M parameters) that sees structured state — five frames of ego and up to 96 other agents at 5 Hz, up to 256 local map tokens, traffic-light states, and goal points — and outputs bounded Beta distributions over longitudinal jerk and tire steering-angle rate, executed by a kinematic bicycle model. Training uses PPO on a reward combining hard safety-event penalties, a one-time goal-arrival bonus, and six soft driving-quality scores multiplied together over a 110-step rollout horizon. To keep late-entering actors from appearing in implausible positions once the ego has deviated from the log, insertion is gated by a scene-consistency check (ego within 5 m and 0.35 rad of the logged pose, no collision with visible actors). The whole rollout loop runs as batched GPU operations, up to 196,608 worlds in parallel across 96 GPUs at 2,048 worlds per rank.

Searching at test time. Rather than training a separate planner, they keep the policy's modal action as candidate 0 and sample additional candidates from the same policy. Each candidate is rolled out for a short horizon (L = 5) with background actors extrapolated by a constant-turn-rate-and-acceleration model, scored using the same reward, discount factor, and critic as PPO, and the best candidate replaces the modal action only if it beats it by a margin δ = 0.03. More candidates buy parallel computation; a longer horizon buys sequential computation.

Learning to see (DriveVFM). Instead of annotating detection, segmentation, lane, and depth tasks, they match the frozen features of four foundation models — DINOv3, SigLIP2, SAM, and Depth Anything V2 — with asymmetric supervision (DINOv3 supervises both summary and patch tokens, SigLIP2 only summary, SAM only patch; Depth Anything V2's visual features are distilled rather than its depth predictions, which performed better in their experiments). Summary features use cosine distance, patch features use mean-squared error, and PHI-S standardizes each teacher's patch features to equalize their influence. Pretraining is two-stage: 256×256 for 600K iterations, then 512×512 for 200K iterations, both at global batch size 2,048, mixing LAION-2B, ImageNet-21K, and SA-1B with driving images from OpenDV, nuPlan, and Waymo. The foundation models and adaptors are discarded afterward.

Turning the teacher into a camera-only planner (DriveZero). The driving log supplies the bridge: each frame pairs multi-view images with the structured state, so a teacher trajectory rolled out from the state can supervise a student looking at images. DriveZero takes four camera views, ego kinematics, and a navigation command derived from the teacher's goal point; a LoRA-finetuned DriveVFM encodes each image; register tokens compress the features into scene tokens; learnable trajectory queries are decoded into 64 proposals; and a separate scoring decoder predicts the six PDM score components used to rank proposals at inference. Only the closest proposal contributes to the trajectory loss (winner-takes-all), and candidate trajectories are detached before scoring so generation and ranking are trained separately. Training is open-loop on logged frames — the student never enters an interactive environment and no rendering is required. Because the teacher is goal-conditioned, the same scene can be queried with augmented routes, producing several valid supervision targets per frame.

Why This Matters

Impact on research. The paper challenges an assumption that runs through most end-to-end driving work: that the recorded human trajectory must be the supervision signal. It offers a concrete recipe — privileged closed-loop RL teacher, annotation-free vision foundation model distillation, open-loop student distillation — and shows on three benchmarks that the resulting system matches or exceeds both the human driver and imitation-based counterparts. It also demonstrates a test-time scaling behavior for driving policies: a fixed checkpoint improves monotonically with candidate budget, from 93.01 at the modal action to 93.57 at 64 candidates. The 5.70M-parameter teacher is also notable for its small size relative to the behavior it reaches.

Real-world applications:

• Camera-only autonomous driving stacks, where the deployed planner must work from images without LiDAR or ground-truth symbolic inputs.
• Large-scale data generation for driving: the RL teacher can be queried under augmented intents to produce diverse, goal-consistent trajectories that logged data does not contain.
• Simulation-based training pipelines for planning policies, including the use of out-of-distribution simulated scenes (237K from SimScale in this work) to scale supervision.
• Robustness testing and validation, since the reward/critic machinery supports test-time action search that can be applied to an existing checkpoint without retraining.

Industry relevance. The work comes from Xiaomi EV's AD & Robotics L3 Team and reports self-play as the configuration used for real-world deployment, indicating the system is intended for production autonomous driving rather than benchmark performance alone. The decomposition into pretraining in separate regimes, combined with the fact that no task-specific perception labels or human trajectories are needed, targets the two scarcest resources in industrial AD development: annotated perception data and diverse, safety-critical driving behavior.

Future Directions

• Closing the loop for the deployed student. DriveZero is trained open-loop on logged frames and evaluated on pseudo closed-loop benchmarks (NAVSIMv1, NAVSIMv2) plus a zero-shot true closed-loop benchmark (HUGSIM). Whether the camera-only policy benefits from closed-loop interaction of its own, rather than only from teacher rollouts, remains open.

• Quantifying when self-play pays off. The paper reports that self-play's benefit on nuPlan is small and attributes this to the benchmark's IDM-driven background traffic, but uses self-play for deployment. Measuring the deployment-time effect, and designing benchmarks that reward natural interaction, is a natural next step.

• Better test-time search and evaluation. The critic and background-actor extrapolation are both approximate, which is why a switching margin is required. Improving either, and understanding the limits of test-time scaling beyond 64 candidates, could extend the gains seen on the non-reactive splits.

• Transferability of the perception backbone and the recipe. The ablations on goal augmentation and on the cumulative contribution of each foundation model are reported only qualitatively in the provided content. Whether the same recipe extends to other sensors, driving datasets, or embodiments beyond nuPlan/NAVSIM/HUGSIM scenes is not established.

Target Audience

Researchers and engineers working on end-to-end autonomous driving, reinforcement learning for control, and vision foundation model distillation. It is most useful to readers who already understand PPO, privileged-policy distillation, and the nuPlan/NAVSIM evaluation protocols, and who want a concrete design for removing human trajectory supervision from the training loop. Practitioners building production AD stacks will find the system-level decomposition and the test-time search mechanism most directly applicable.

Note on scope of this summary: the provided paper content is truncated. Values for the HUGSIM route-completion metric, the remainder of the NAVSIMv2 results table, the DriveRL per-component metrics (Appendix A.5.1, Table A5), the self-play tables (A6, A7), the ViT-S and ViT-B backbone results (Appendix B.2), and the numerical ablation results are not present in the content above and are therefore not reported here.

Authors’ abstract

Most end-to-end autonomous-driving systems learn by imitating human driving logs, leaving their learned behavior constrained by the quality and behavioral coverage of the recorded trajectories. This report presents DriveZero, an end-to-end system that learns driving behavior beyond human demonstrations. It decomposes driving into a perception model and an action model, pretrains each in the regime best suited to it, and combines them into one end-to-end planner. The two models call for different learning recipes: perception must understand the world, and benefits from massive and diverse visual data; action must interact with it, and requires closed-loop feedback. On the action side, we introduce DriveRL, a mixed-agent closed-loop reinforcement-learning framework. It converts real driving logs into interactive worlds, where a privileged teacher policy is trained with PPO through closed-loop rollouts. For the perception model, DriveVFM consolidates multiple frozen vision foundation models, including DINOv3, SigLIP2, SAM and Depth Anything V2, into a single backbone from raw images alone, requiring no task-specific annotations. DriveZero then unifies the two: a camera-only planner that distills the frozen DriveRL teacher through its rolled-out trajectories. The goal-conditioned teacher can moreover be queried under augmented driving intents, yielding diverse, goal-consistent supervision that logged data cannot provide. On nuPlan, DriveRL with value-guided test-time action search achieves a mean score of 93.57 across the Val14, Test14-hard, and Test14-random community splits in both non-reactive and reactive modes, exceeding the Log-Replay expert on all three splits. DriveZero achieves state-of-the-art performance on NAVSIMv1, NAVSIMv2 and the closed-loop HUGSIM benchmark without any human trajectory supervision.