Pandar128 dataset for lane line detection

AI summary

Overview

Research area: Autonomous driving perception — specifically LiDAR-based lane line detection and benchmark dataset construction.

Technical level: Intermediate. The paper is readable without deep prior knowledge of 3D perception, but familiarity with bird's-eye-view (BEV) representations, point cloud segmentation, and standard detection metrics (F1, precision, recall) helps considerably. The evaluation metric section includes formal notation.

Scope: The paper releases Pandar128, a large 128-beam LiDAR dataset with lane-line annotations, odometry, and full sensor calibration, and accompanies it with a lightweight baseline pipeline (SimpleLidarLane) and a new polyline-based evaluation metric (IAM-F1).

What This Paper Is About

Lane line detection is essential for driver assistance systems such as Lane-Keeping Assist, and while camera-based methods work, they struggle in low light, glare, and heavy shadows, and they lose geometric accuracy when lanes are projected from the image plane into 3D vehicle coordinates. LiDAR avoids these problems by capturing 3D geometry directly, but very few public datasets provide lane-marking annotations in LiDAR point clouds. This paper addresses that gap by publishing a large 128-beam LiDAR lane detection dataset, a simple baseline method that runs on it, and a standardized metric for comparing polyline predictions against ground truth.

Key Contributions

1. Pandar128, the first lane detection dataset collected with a 128-beam LiDAR. The paper states this beam count is higher than the 64-beam LiDARs used by comparison datasets such as KITTI-360, K-Lane, Waymo, and LiSV-3DLane.

2. A large annotated corpus of camera and LiDAR data. The abstract and contribution list report over 52,000 camera frames and 34,829 LiDAR frames (52,200 images specifically), while Section 3.2 reports 46,802 image frames and 34,831 LiDAR and odometry frames. Section 3.2 also describes 29 driving traces of roughly 60 seconds each, with approximately 1,800 video frames and 1,200 LiDAR scans per trace. The comparison table cites 35k frames for the dataset. The paper does not reconcile these differing counts.

3. Full sensor calibration plus synchronized odometry. Intrinsics, extrinsics, distortion coefficients, and relative transformation matrices between consecutive frames are included, enabling projection, sensor fusion, temporal accumulation, and map-level lane modeling. The authors state this combination is not available in prior lane detection datasets.

4. SimpleLidarLane, a modular baseline pipeline combining BEV segmentation, anisotropic scaling, DBSCAN clustering, and RANSAC polyline fitting.

5. IAM-F1 (Interpolation-Aware Matching F1), a new polyline-based evaluation metric that performs interpolation-aware lateral matching in BEV space with an explicitly defined tolerance of 0.2 m.

Main Findings

• Segmentation alone is the weakest of the three metrics. Averaged over six test traces, semantic segmentation F1 was 0.7152, rasterized (meshgrid) polyline F1 was 0.8095, and the proposed IAM-F1 was 0.8486.

• IAM-F1 scored highest on five of six test traces. Per-trace IAM-F1 values were 0.9136, 0.9093, 0.9263, 0.9093, 0.8384, and 0.5945, against segmentation scores of 0.8434, 0.7793, 0.7419, 0.7931, 0.7139, and 0.4195 respectively.

• Trace 1110_131951_005 is a clear outlier and the one case where meshgrid beat IAM-F1 (0.6118 versus 0.5945), with all three metrics dropping sharply compared to the other traces.

• Reported average performance figure is internally inconsistent. Section 5.3 states the pipeline achieves an average IAM-F1 of 80.95% on the test set, but the IAM-F1 average in Table 3 is 0.8486; 0.8095 corresponds to the meshgrid polyline column. Readers should treat the 80.95% figure with caution.

• Performance degrades in complex environments. Segmentation performs well on structured highway scenes but degrades at urban–highway transitions, in merging zones with worn or missing markings, in sparse rain returns, and on highly curved road geometries.

• Range limits under adverse conditions. The method reliably reconstructs lane lines up to 40 m, and up to 25 m under adverse weather.

• For context, the paper cites K-Lane's own reported F1 of 82.1% on the K-Lane dataset — a different dataset, so this is background rather than a head-to-head comparison.

Methodology in Plain English

The researchers mounted a Pandar128 LiDAR on a vehicle roof, alongside a 2896×1876-pixel front camera behind the windshield and a GPS/IMU unit, and drove 29 traces in Germany, mostly on highways in sunny, low-traffic conditions with a small number of rain, construction, and mid-traffic sequences.

For annotation, they provided two label types: per-point semantic segmentation of the point cloud (background = 0, white lane line = 1, yellow lane line = 2) and much cheaper polyline annotations stored as ordered lists of (x, y, z) coordinates. Annotation rules cover only visible lane lines in the ego direction, exclude opposite-direction lanes behind physical barriers, and include visible exit and acceleration lanes. They measured lane curvature using the Pearson correlation between x and y coordinates of each polyline, and measured scene complexity by counting lane lines per frame.

For the baseline, they flatten the point cloud into a top-down grid covering 0–40 m ahead and ±15 m to the sides at 5 cm per cell. A U-Net segmentation network with a ResNet-18 backbone (Section 4.1) — described as U-Net++ with an ImageNet-pretrained ResNet-18 encoder in Section 5.1 — predicts lane pixels. The output is stretched along the forward axis and squeezed laterally to pull lanes apart, then DBSCAN groups the pixels into instances, and RANSAC fits a polyline to each cluster. Training used focal loss with default settings, the Adam optimizer with an initial learning rate of 0.01, ReduceLROnPlateau scheduling, early stopping on validation IoU, batch sizes of 2 and 1, for up to 400 epochs on an NVIDIA RTX 4060 (8 GB VRAM).

The new IAM-F1 metric works on 2D (x, y) polylines only, ignoring height. For each ground-truth point, it interpolates the predicted polyline at the same longitudinal position and measures the lateral gap; gaps under 0.2 m count as true positives, larger gaps or points outside the interpolation domain count as false negatives, and the process repeats in reverse to count false positives.

Why This Matters

Impact on research: The paper argues that algorithms for lane detection far outnumber the datasets available to evaluate them, and that only three datasets (CULane, TuSimple, and K-Lane) had been used for benchmarking. Waymo and LiSV-3DLane are described as offering LiDAR lane annotations but not positioned as primary lane detection benchmarks. Providing a larger, 128-beam dataset with calibration and odometry gives the field a richer testbed for fusion and temporal methods, and IAM-F1 offers a metric that avoids the quantization artifacts of fixed-resolution raster comparison.

Real-world applications:

• Lane-Keeping Assist, which the paper notes has been legally required in all new cars sold in the EU starting from July 2024.
• Adaptive Cruise Control and other ADAS functions that consume lane geometry.
• Path planning that needs accurate lane position and curvature, where lateral accuracy in the road plane is what matters.
• Robust perception in rain, glare, low illumination, and dense traffic, where the paper argues camera-only pipelines are weakest.

Industry relevance: Automotive manufacturers and tier-one suppliers building production driver-assistance features need lane perception that holds up across weather and lighting. A public dataset with calibration and odometry lowers the barrier for training sensor-fusion and temporal-accumulation models, and the modular, interpretable baseline is directly useful for prototyping and for isolating whether failures come from perception, clustering, or fitting.

Future Directions

• Closing the robustness gap. The paper highlights the need for improved handling of occlusions, severe weather, and highly curved lane structures, which are exactly the failure modes seen in the qualitative results.

• Reconciling the scope limitations of the dataset itself. The authors acknowledge a strong focus on highway driving under sunny, low-traffic conditions; expanding rain, construction, and mid-traffic coverage would make robustness claims more measurable.

• Reconciling the reported statistics. The frame counts differ between the abstract, the contribution list, Section 3.2, and Table 1, and the headline accuracy figure in Section 5.3 does not match the corresponding column in Table 3. A corrected, consistent accounting would strengthen reproducibility.

• Exploring the calibration and odometry for temporal and fusion work. The paper positions these as enablers for multi-frame alignment, sensor fusion, and map-level lane modeling, but the experiments only train the single-frame segmentation component — leaving the main advertised capabilities of the dataset untested in this paper.

Target Audience

Researchers and graduate students working on autonomous driving perception, LiDAR point clouds, and lane detection; engineers building ADAS or Lane-Keeping Assist features who need a benchmark with calibration and odometry; and practitioners who want a simple, interpretable baseline pipeline they can adapt rather than a large end-to-end model. Reviewers and dataset builders will also find the IAM-F1 metric and the annotation guidelines useful as a template.

Authors’ abstract

We present Pandar128, the largest public dataset for lane line detection using a 128-beam LiDAR. It contains over 52,000 camera frames and 34,000 LiDAR scans, captured in diverse real-world conditions in Germany. The dataset includes full sensor calibration (intrinsics, extrinsics) and synchronized odometry, supporting tasks such as projection, fusion, and temporal modeling. To complement the dataset, we also introduce SimpleLidarLane, a light-weight baseline method for lane line reconstruction that combines BEV segmentation, clustering, and polyline fitting. Despite its simplicity, our method achieves strong performance under challenging various conditions (e.g., rain, sparse returns), showing that modular pipelines paired with high-quality data and principled evaluation can compete with more complex approaches. Furthermore, to address the lack of standardized evaluation, we propose a novel polyline-based metric - Interpolation-Aware Matching F1 (IAM-F1) - that employs interpolation-aware lateral matching in BEV space. All data and code are publicly released to support reproducibility in LiDAR-based lane detection.