WorldRFT: Latent World Model Planning with Reinforcement Fine-Tuning for Autonomous Driving

AI summary

Overview

Research area: Robotics / end-to-end autonomous driving — specifically latent world models trained with self-supervised learning and refined with reinforcement learning for trajectory planning. Technical level: Advanced (assumes familiarity with latent world models, deformable attention/convolution, imitation learning, and policy-gradient RL such as GRPO). Scope: This paper proposes WorldRFT, a planning-oriented latent world model that fuses VGGT geometric priors, decomposes planning into hierarchical subtasks, and applies GRPO-based reinforcement fine-tuning, evaluated on the open-loop nuScenes and closed-loop NavSim benchmarks.

What This Paper Is About

Latent world models learn scene representations by predicting future latent states through temporal self-supervision, which avoids the need for expensive perception annotations. The problem is that these representations are optimized for reconstruction, not for planning, so they are misaligned with what a driving policy actually needs. WorldRFT's goal is to realign representation learning with planning and to add explicit safety objectives through reinforcement learning, rather than relying only on imitating expert trajectories.

Key Contributions

1. A planning-oriented latent world modeling paradigm. The paper describes itself as the first to deeply align representation learning with planning tasks via spatial-geometric prior fusion (a frozen VGGT spatial encoder), hierarchical planning interaction, and local-aware refinement.
2. A reinforcement fine-tuning phase for safety. The authors introduce a GRPO-based phase with Gaussianized trajectory modeling and collision-aware rewards, shifting the policy from passive behavior cloning toward proactive collision avoidance.
3. State-of-the-art results on both benchmark types. WorldRFT reports SOTA on open-loop nuScenes and closed-loop NavSim, with the paper reporting the lowest collision rate among all evaluated nuScenes methods.
4. Architectural components with measured individual effects. The Spatial-aware World Encoder, Hierarchical Planning Refinement module (target region localization, spatial path planning, temporal trajectory prediction), and local-aware iterative refinement are each ablated on nuScenes.

Main Findings

• nuScenes trajectory accuracy and safety improve sharply over the LAW baseline. Average displacement error (L2) drops 21% (0.61 m → 0.48 m) and collision rate drops 83% (0.30% → 0.05%).
• NavSim PDMS improves by 3.2 points with camera-only input (84.6 → 87.8), approaching the LiDAR-based SOTA method DiffusionDrive at 88.1 — a gap of 0.3 points.
• The paper reports the lowest collision rate among all evaluated nuScenes methods, including perception-based approaches; WorldRFT (with RFT) reaches 0.05% average collision rate versus DiffusionDrive's 0.08%.
• NavSim safety-critical sub-metrics improve over LAW: No At-fault Collision 96.4 → 97.8, Time-to-Collision 88.7 → 94.0, and Drivable Area Compliance 95.4 → 96.8. The paper notes 96.8 DAC is the highest among all methods listed, including LiDAR-based ones.
• Reinforcement fine-tuning trades a small accuracy cost for large safety gains. Comparing "Ours (without RFT)" to "Ours (with RFT)" on nuScenes, collision rate falls 66.7% (0.15 → 0.05) while L2 rises slightly from 0.47 to 0.48.
• VGGT geometric priors help consistently. Comparing ablation ID 4 with ID 8, adding VGGT reduces L2 error by 7.7% (0.52 → 0.48) and collision rate by 37.5% (0.08 → 0.05).
• Hierarchical task decomposition drives the largest accuracy gain. Going from the baseline (ID 1) to ID 4 reduces L2 error by 11.9% (0.59 → 0.52) and collision rate by 50.0% (0.16 → 0.08).
• Local-aware iterative refinement gives further gains. Comparing ID 7 (without refinement) to ID 8 (with refinement), L2 error falls 4.0% (0.50 → 0.48) and collision rate falls 16.7% (0.06 → 0.05).
• Qualitative attention maps shift toward planning-critical agents. The visualizations show WorldRFT attending to surrounding vehicles that the baseline largely overlooks, and RFT trajectories that keep safer distances than pre-trained trajectories.

Methodology in Plain English

The pipeline follows scene understanding → planning decisions → safety optimization.

1. Spatial-aware World Encoder. Surround-view images are encoded by an image backbone (ResNet-50 on nuScenes, ResNet-34 on NavSim). Semantic supervision comes from pseudo-labels generated by Grounded-SAM. Separately, a frozen VGGT vision-geometry foundation model extracts 3D tokens, which are injected into the 2D features through a single cross-attention layer where VGGT tokens serve as keys and values. This yields a unified latent representation with geometric awareness but no explicit depth input or 3D annotation.
2. Hierarchical Planning Refinement. Instead of one global planning query, three dedicated queries handle three parallel subtasks: target region localization, spatial path planning, and temporal trajectory prediction. Target regions are modeled as Laplace distributions (a center and a scale parameter) rather than single points, with the scale acting as a signal of scene uncertainty. Spatial paths are supervised with points sampled at fixed 2 m intervals (30 points on nuScenes, 50 on NavSim), making them purely geometric; temporal trajectories use fixed 0.5-second intervals over 3 seconds (6 points). The queries exchange information via self-attention so they stay mutually aware.
3. Local-aware Iterative Refinement. Over K = 3 iterations, the current plan is encoded into a state vector, trajectory points are projected into the latent feature map via camera parameters, local features are sampled with deformable convolution, and everything is fused with the global planning intention and the uncertainty scale parameter. Each iteration predicts a residual offset applied with a step size of 0.1, refining trajectories incrementally rather than predicting them once.
4. Reinforcement Fine-Tuning. Trajectories are recast as Gaussian distributions so the policy can be sampled. A group of 10 trajectories is drawn, each point receives a normalized relative reward (collision = −1, no collision = 0), and advantages are accumulated over subsequent timesteps to account for error propagation. The GRPO objective with a clipping term and KL regularization against a reference policy updates the model. Training also uses a reference loss to stay close to the pre-trained mean trajectory and a maximum-entropy loss to avoid premature convergence.

Why This Matters

Research impact: The paper argues that reconstruction-oriented latent world models are structurally mismatched with planning, and shows that a planning-aligned representation plus RL fine-tuning can beat perception-annotation-heavy pipelines on safety metrics. If it holds up, that reframes how self-supervised driving representations should be designed and evaluated — moving the field from "predict the future well" to "predict the future in a way that supports safe decisions."

Real-world applications (implications drawn from the paper's camera-only results):

• Camera-only autonomous driving stacks, which reduce sensor cost and integration complexity relative to LiDAR-based systems; WorldRFT reports 87.8 PDMS with camera input alone.
• Collision-avoidance layers for production planning modules, since the RFT stage can be applied on top of a pre-trained policy to sharpen safety behavior at a small accuracy cost.
• Autonomous fleets and robotaxi deployment, where the paper's safety-critical metrics (no at-fault collision, time-to-collision) are the operational metrics that matter most.
• Scalable training pipelines that avoid costly 3D perception annotation, since the world model is pretrained with self-supervision on future-world prediction.

Industry relevance: The work involves authors from the Institute of Automation, CAS, UCAS, and Li Auto, and it is positioned around deployment potential. Two of the paper's claims are directly relevant to commercialization: competitive performance without LiDAR, and a fine-tuning stage that improves explicit safety objectives rather than just imitation fidelity. The paper does not report on-road or production deployment results.

Future Directions

• Closing the 0.3-point NavSim gap to LiDAR-based DiffusionDrive (87.8 vs. 88.1) with camera-only input, and testing whether adding LiDAR closes it entirely or whether the latent world representation saturates.
• Reconciling the RFT accuracy trade-off. Fine-tuning improves collision rate (0.15 → 0.05) but slightly worsens L2 (0.47 → 0.48); whether reward shaping can improve both simultaneously is left open.
• Scaling and generalization. The paper mentions a scalability analysis in the supplementary material, but the truncated content does not report its results; how the method behaves on larger datasets and rare safety-critical scenarios remains untested here.
• Reducing reliance on auxiliary models. The pipeline depends on a frozen VGGT encoder and on Grounded-SAM pseudo-labels for semantic supervision; replacing or jointly training these components could simplify the system and remove inherited biases.

Target Audience

Researchers and engineers working on end-to-end autonomous driving, latent world models, and reinforcement learning for decision-making. It is best suited to readers already comfortable with attention-based planning architectures and policy optimization, since the paper's value is in how it restructures representation learning and adds RL fine-tuning, not in introductory exposition. Practitioners evaluating camera-only stacks for deployment will find the benchmark comparisons and the safety-metric deltas most directly useful.

Authors’ abstract

Latent World Models enhance scene representation through temporal self-supervised learning, presenting a perception annotation-free paradigm for end-to-end autonomous driving. However, the reconstruction-oriented representation learning tangles perception with planning tasks, leading to suboptimal optimization for planning. To address this challenge, we propose WorldRFT, a planning-oriented latent world model framework that aligns scene representation learning with planning via a hierarchical planning decomposition and local-aware interactive refinement mechanism, augmented by reinforcement learning fine-tuning (RFT) to enhance safety-critical policy performance. Specifically, WorldRFT integrates a vision-geometry foundation model to improve 3D spatial awareness, employs hierarchical planning task decomposition to guide representation optimization, and utilizes local-aware iterative refinement to derive a planning-oriented driving policy. Furthermore, we introduce Group Relative Policy Optimization (GRPO), which applies trajectory Gaussianization and collision-aware rewards to fine-tune the driving policy, yielding systematic improvements in safety. WorldRFT achieves state-of-the-art (SOTA) performance on both open-loop nuScenes and closed-loop NavSim benchmarks. On nuScenes, it reduces collision rates by 83% (0.30% -> 0.05%). On NavSim, using camera-only sensors input, it attains competitive performance with the LiDAR-based SOTA method DiffusionDrive (87.8 vs. 88.1 PDMS).