LaneDiffusion: Improving Centerline Graph Learning via Prior Injected BEV Feature Generation

AI summary

Overview

Research area: Computer vision for autonomous driving — specifically online lane centerline graph learning (vectorized lane detection plus topology inference) from surround-view camera images.

Technical level: Advanced. The paper builds on denoising diffusion probabilistic models, BEV feature construction, deformable attention, and transformer-based encoders, and it reports point-level and segment-level map metrics that require familiarity with the lane graph literature.

Scope: The paper proposes LaneDiffusion, a diffusion-based framework that generates lane centerline priors at the BEV feature level rather than generating vectorized centerlines directly, and reports state-of-the-art results on nuScenes and Argoverse2.

What This Paper Is About

Lane centerline graphs are needed so a self-driving car knows where it can drive and how lanes connect, but the lanes themselves are often invisible, occluded, or visually ambiguous. Almost all prior methods for this task are deterministic — they produce one answer and cannot model uncertainty — while generative approaches have been largely unexplored here. LaneDiffusion's goal is to insert the probabilistic modeling power of diffusion into an existing BEV-based lane detection pipeline so that missing or occluded centerlines can be recovered more reliably.

Key Contributions

1. An end-to-end generative framework for centerline graph learning. LaneDiffusion is described as the first approach to use diffusion models to generate lane centerline priors at the BEV feature level, instead of directly predicting vectorized centerlines.

2. Lane Prior Injection Module (LPIM). A module that encodes lane centerline ground truth as prior knowledge and injects it into BEV features to construct the diffusion target. Ground truth is used only to build this target, so it is not needed at inference and there is no ground truth leakage concern at test time.

3. Lane Prior Diffusion Module (LPDM). A DDPM-based module that learns to model the prior-injected BEV feature. Its Markov chain transitions from the prior-injected BEV feature back to the original BEV feature (rather than to Gaussian white noise), which substantially reduces the number of sampling steps required. It uses a Swin-transformer-based UNet as the denoising network.

4. A lane prior refinement mechanism plus extensive validation. The generated feature is fused with the original feature before being passed to lane decoders, and the method is evaluated on nuScenes and Argoverse2 with both fine-grained point-level and segment-level metrics.

Main Findings

• nuScenes point-level results (Table 1, 24 epochs): LaneDiffusion reaches GEO F1 58.9 (+4.2 over CGNet's 54.7), TOPO F1 46.8 (+4.6 over 42.2), JTOPO F1 38.8 (+4.7 over 34.1), APLS 37.1 (+6.4 over 30.7), and SDA 10.6 (+1.8 over 8.8).

• nuScenes segment-level results (Table 2, 24 epochs): IoU 58.6 (+2.3 over CGNet's 56.3), mAP_cf 41.6 (+6.4 over 35.2), DET_l 28.8 (+6.8 over 22.0), and TOP_ll 3.4 (+2.1 over 1.3).

• Argoverse2 validation results (Table 3, 6 epochs): TOPO F1 47.0 (+2.5 over CGNet's 44.5), JTOPO F1 36.4 (+1.8 over 34.6), APLS 25.8 (+2.2 over 23.6), SDA 14.4 (+0.7 over 13.7), and TOP_ll 1.8 (+1.3 over 0.5).

• Baselines compared against: On nuScenes the comparison set is STSU (200 epochs), HDMapNet (30 epochs), VectorMapNet, TopoNet, MapTR and CGNet (24 epochs); on Argoverse2 it is TopoNet, MapTR and CGNet (6 epochs).

• Diffusion step count matters (Table 4): T = 15 with the sampling paradigm gives TOPO F1 46.8, JTOPO F1 38.8, APLS 37.1 and SDA 10.6 — improvements of 2.0%, 2.4%, 4.0% and 1.4% respectively. Increasing T to 30 lowered inference efficiency and gave slightly worse performance (TOPO F1 46.0, APLS 36.3, SDA 10.2).

• Training paradigm trade-off (Table 4): At T = 5, the "Overall" paradigm (full denoising with loss on intermediate features at every timestep) scored TOPO F1 45.6 and APLS 35.5, beating the "Sampling" paradigm at T = 5 (44.8 and 33.1). The authors state they still use the Sampling paradigm for larger T because of GPU memory limits.

• Refinement ablation (Table 5): No refinement gives TOPO F1 43.9 and APLS 33.3; Concat&FC gives 45.6 / 35.7; Concat&ED gives 46.8 / 37.1; Add&FC gives 45.3 / 35.7; Add&ED gives 46.8 / 36.9. Encoder-decoder refinement beats a plain 1×1 convolution, and concatenation versus addition performs similarly. Cross-attention-based fusion was not explored due to GPU memory limits.

• Qualitative behavior: The paper states that under various weather and lighting conditions on nuScenes, LaneDiffusion captures missing lane structures that the deterministic CGNet baseline fails to detect.

• Not reported: The paper does not report inference latency, frames per second, parameter counts, or model size. It only states, in the conclusion, that future work will target real-time performance and a more lightweight solution.

Methodology in Plain English

The pipeline builds on an existing strong baseline, CGNet, which turns surround-view camera images into a bird's-eye-view feature map and then decodes lane segments and their connections from it. LaneDiffusion adds two new pieces on top of this.

First, the Lane Prior Injection Module takes ground-truth lane centerlines, samples each one into N fixed points, embeds those points with sinusoidal positional embeddings, encodes them with a transformer encoder, and cross-attends them into the BEV features produced at each stage of the BEV constructor. The result is a BEV feature map that carries lane information — this becomes the target that the diffusion model must learn to produce.

Second, the Lane Prior Diffusion Module learns to turn the original BEV feature into that prior-injected BEV feature. Crucially, it does not start from random noise. Instead of the usual "add noise until the signal becomes Gaussian" scheme, it gradually shifts the residual between the original feature and the prior-injected feature over a short Markov chain, so the model starts from the original BEV feature and only has to learn the difference. That is why 15 steps suffice instead of the hundreds or thousands usually needed. Training minimizes a weighted squared-error loss between the network's predicted target and the true prior-injected feature.

The generated feature is then merged back with the original BEV feature through a refinement encoder-decoder, and the merged feature is fed to the same kind of lane decoder the baseline uses to output vectorized centerline segments and their connectivity.

Training happens in three stages: train LPIM with the baseline's collection of lane losses (classification, polyline point distance, topology, direction, Bézier consistency, and junction points); freeze LPIM and train LPDM with the diffusion loss; then freeze LPDM in sampling mode and train the lane decoder. At inference, because the generation is stochastic, the LPDM sampling is run three times and the generated features are averaged for stability.

Why This Matters

Impact on research. The paper argues that generative modeling has been underexplored for centerline graph learning and that deterministic methods like CGNet struggle with occlusions and ambiguous visual cues. By moving diffusion to the BEV feature level instead of the vectorized output level, it sidesteps a key weakness of earlier generative map work such as PolyDiffuse, which the authors say must predetermine the number of vectorized outputs and depends on a pre-trained external model for initialization. The modular design is presented as a drop-in add-on for any BEV-feature-based architecture, which means the idea could transfer to other BEV map-perception systems.

Real-world applications:

• Path planning and control for self-driving vehicles, which the paper identifies as the primary use of centerline graphs.
• Online high-definition map construction and map maintenance from camera data.
• Robust lane perception in rain, night, and other conditions where lanes are occluded or poorly lit, as illustrated in the qualitative comparisons.
• Any BEV-based driving stack that needs improved topology reasoning about how lane segments connect at junctions.

Industry relevance. Three of the authors are affiliated with Baidu Inc., and the work was partly done during an internship at Baidu, which points to direct relevance for commercial autonomous driving and mapping pipelines. The add-on nature of the design lowers adoption cost: an existing BEV model can keep its backbone and decoder and insert LPIM and LPDM.

Future Directions

• Real-time performance and model size. The conclusion explicitly states that future work will focus on improving real-time performance and reducing scale toward a more lightweight solution; no latency or throughput numbers are reported in this paper.
• Scaling the training paradigm with more diffusion steps. The ablation shows the "Overall" training paradigm beats "Sampling" at T = 5, and the authors suggest performance may improve further with larger T, but they could not test this because of GPU memory limits.
• Stronger feature fusion. The paper notes that cross-attention-based fusion of generated and original features could outperform concatenation or addition, but it was not explored due to GPU memory constraints.
• Generalization beyond the tested benchmarks. Whether the BEV-level diffusion prior helps other map elements, other BEV backbones, or datasets beyond nuScenes and Argoverse2 is left open.

Target Audience

Researchers and engineers working on autonomous driving perception, online HD map construction, and vectorized lane graph learning, particularly those already using BEV-based architectures such as CGNet. It also suits readers interested in applying diffusion models to structured perception tasks rather than image synthesis. Because the method is evaluated with specialized lane graph metrics (GEO F1, TOPO F1, JTOPO F1, APLS, SDA, IoU, mAP_cf, DET_l, TOP_ll) and heavy diffusion notation, some background in the lane graph literature and in diffusion models is needed to follow the methodology in detail.

Authors’ abstract

Centerline graphs, crucial for path planning in autonomous driving, are traditionally learned using deterministic methods. However, these methods often lack spatial reasoning and struggle with occluded or invisible centerlines. Generative approaches, despite their potential, remain underexplored in this domain. We introduce LaneDiffusion, a novel generative paradigm for centerline graph learning. LaneDiffusion innovatively employs diffusion models to generate lane centerline priors at the Bird's Eye View (BEV) feature level, instead of directly predicting vectorized centerlines. Our method integrates a Lane Prior Injection Module (LPIM) and a Lane Prior Diffusion Module (LPDM) to effectively construct diffusion targets and manage the diffusion process. Furthermore, vectorized centerlines and topologies are then decoded from these prior-injected BEV features. Extensive evaluations on the nuScenes and Argoverse2 datasets demonstrate that LaneDiffusion significantly outperforms existing methods, achieving improvements of 4.2%, 4.6%, 4.7%, 6.4% and 1.8% on fine-grained point-level metrics (GEO F1, TOPO F1, JTOPO F1, APLS and SDA) and 2.3%, 6.4%, 6.8% and 2.1% on segment-level metrics (IoU, mAP_cf, DET_l and TOP_ll). These results establish state-of-the-art performance in centerline graph learning, offering new insights into generative models for this task.