Scenario-Independent Criticality Assessment and Prediction for Vulnerable Road Users in Autonomous Driving

AI summary

Overview

Research area: Autonomous driving safety — perception criticality assessment and prediction for vulnerable road users (VRUs), combining traffic safety metrics with machine learning.

Technical level: Advanced. The paper assumes familiarity with autonomous driving safety metrics (time-to-collision family, RSS, reachability analysis) and standard ML classification workflow.

Scope: The paper introduces a pedestrian-focused criticality metric, a large labeled criticality dataset, and a scenario-independent ML framework for predicting which detected objects are safety-critical.

What This Paper Is About

Autonomous vehicles need to know which detected objects actually matter for safety — a pedestrian stepping into the road is critical, a car driving away behind the ego vehicle is not. Existing criticality metrics were built mostly for car-to-car interactions and are tuned to specific scenarios, so they generalize poorly and largely ignore pedestrians and cyclists, whose motion is far less predictable.

This paper addresses both gaps: it proposes a criticality metric designed around VRU movement uncertainty, and a machine-learning framework that predicts criticality for any traffic participant class without scenario-specific tuning.

Key Contributions

1. A perception criticality dataset — the first with object-dynamics-aware criticality labels, built on the synthetic DeepAccident dataset. It contains 161,744 frames and 5,461,327 labeled objects, with per-object class, position, dimensions, yaw, velocity, LiDAR point counts, and camera visibility.

2. A VRU-centric criticality metric (CVRU) — combines pedestrian motion/behavior models with the Space Occupancy Index, using intersecting reachable-space polygons to decide whether a VRU is critical.

3. A scenario-independent criticality prediction framework — applies supervised and unsupervised statistical models to the dataset features, letting criticality be learned from data instead of hand-tuned per-scenario rules, and working across all object classes.

4. A dual labeling strategy — criticality ground truth generated both by aggregating established metrics (TTC, CIF, RSS) and by trajectory-based reachability analysis with kinematic constraints.

Main Findings

• VRU metric beats state of the art by 50%: CVRU reached an F1-score of 0.5687 on the pedestrian class versus 0.3865 for the best competing metric (RSS) — a relative improvement of about 50%. Precision was 0.9587 and recall 0.4494.

• Existing metrics fail on pedestrians: TTC scored 0.99 precision but only 0.029 recall (F1 = 0.056), reflecting near-total blindness to non-collision-but-critical pedestrians. CIF (0.077), MTTC (0.139), TTA (0.164), and SACRED (0.161) all performed poorly.

• Supervised prediction dominates: Over all object classes, Random Forest achieved the best result with precision 0.9303, recall 0.9415, and F1 0.9359 — roughly a 275% improvement over the best SOTA metric (RSS at 0.3545). The abstract cites an F1-score of 0.96.

• Tree-based models outperform linear and unsupervised models: LightGBM (0.9158) and histogram-based gradient boosting (0.9117) closely followed Random Forest, while linear regression reached 0.6836. Unsupervised methods were weak across the board (Gaussian mixture model 0.0922 to DBScan 0.1678).

• Scenario dependence is confirmed: Results reinforce earlier findings that SOTA metrics do not transfer across scenarios, which is the core motivation for replacing handcrafted metrics with learned prediction.

• High recall metrics trade off precision: TTA and SACRED achieve recalls above 0.80 but precisions below 0.10, showing they flag almost everything critical rather than discriminating usefully.

Methodology in Plain English

The researchers start from an existing synthetic accident dataset (DeepAccident) containing sensor-equipped vehicles and surrounding traffic. They split it into 1,920 ego-centric scenarios and generate two sets of criticality labels for every object: one by taking the consensus of three trusted safety metrics, and one by geometric reasoning — checking whether an object's reachable future space overlaps the ego vehicle's planned trajectory within a time horizon.

That reachable space is shaped by physics. Pedestrians get a circular zone shifted slightly forward, sized by their walking speed times the prediction horizon, because they can turn and stop abruptly. Vehicles and cyclists get polygon-shaped zones derived from a maximum steering angle, which itself depends on speed, road friction, and axle distance. A pedestrian on the road within twice the ego vehicle's braking distance — with a minimum zone of 6 meters — is always marked critical.

For the VRU-specific metric, the same reachable-space logic is applied but with pedestrian-specific behavior modeling at an assumed fast-walking speed of 2.8 m/s. If the pedestrian's possible space intersects the vehicle's possible space, the pedestrian is critical.

For prediction, each object is described by a 12-dimensional feature vector (class, position, size, yaw, velocity, LiDAR points, camera visibility). Four unsupervised models (KMeans, DBScan, Isolation Forest, Gaussian mixture) and four supervised models (linear regression, random forest, histogram-based gradient boosting, LightGBM) are trained on an 80/20 split with balanced class weights, Optuna hyperparameter tuning, and decision-threshold scanning to maximize F1.

Why This Matters

Criticality assessment sits between perception and planning in an autonomous driving stack. If the system cannot rank which detections actually threaten safety, it either wastes compute on irrelevant objects or misses the ones that matter. This paper provides both a better metric for the hardest class of road users and a general-purpose learned alternative that removes scenario-specific hand-tuning.

Impact on research:

• Provides a public benchmark and labeled dataset for criticality classification, which previously had no standardized definition or ground truth.
• Shifts criticality estimation from handcrafted analytic metrics toward data-driven prediction, opening a new line of work.
• Gives a VRU-specific formulation that future pedestrian and cyclist safety work can build on directly.

Real-world applications:

• Perception triage in autonomous vehicles, prioritizing detection and tracking resources on safety-critical objects.
• Safety evaluation and validation of perception stacks during development, using F1-style criticality scores instead of generic average precision.
• Pedestrian and cyclist safety systems in advanced driver assistance, including automatic emergency braking and vulnerable-road-user detection.
• Regulatory and type-approval testing, where scenario-independent, explainable criticality measures support safety case arguments.

Industry relevance: Automotive OEMs, Tier 1 suppliers, and autonomous driving software companies need safety metrics that are auditable, generalizable, and computable in real time. The preference for simple, traceable statistical models rather than deep networks fits the explainability constraints of functional safety standards.

Future Directions

• Add temporal information. The current prediction uses single-frame features; incorporating object history could substantially improve recall, which is the weakest element of even the best models.
• Improve recall on VRUs. CVRU's recall of 0.4494 means over half of critical pedestrians are still missed — the primary remaining safety gap.
• Expand and fine-tune the model set. The authors plan additional prediction methods and more tuning, and eventually folding the CVRU metric itself into the prediction pipeline.
• Class-specific analysis. More granular per-object-class evaluation is promised, which matters because pedestrians, cyclists, motorcyclists, and vehicles have very different motion constraints and error profiles.

Target Audience

Researchers and engineers working on autonomous driving safety, perception evaluation, and vulnerable road user protection. Also relevant to practitioners building perception or planning systems who need practical criticality scoring, and to graduate students studying safety metrics, dataset construction, and the intersection of classical traffic-safety theory with machine learning. Readers should already be comfortable with standard criticality metrics and supervised classification concepts.

Authors’ abstract

Increasing safety is the primary objective of automated vehicles. Achieving this goal requires reliable safety metrics that incorporate safety-relevant factors such as object type, velocity, and criticality. A key capability of such metrics is the distinction between critical and non-critical objects, which is addressed through criticality or relevance estimation. Existing criticality metrics are typically designed for specific scenarios and primarily focus on vehicle-to-vehicle interactions. In this paper, we therefore propose a novel criticality metric tailored to vulnerable road users (VRUs), which require special consideration due to their less predictable motion behavior. Furthermore, to avoid the complexity introduced by scenario-specific metrics, we introduce a scenario-independent criticality prediction framework applicable to all traffic participant classes. The effectiveness of both the proposed VRU-centric criticality metric and the criticality prediction framework is evaluated using the DeepAccident dataset, which contains a diverse set of safety-critical traffic scenarios. The proposed VRU-centric criticality metric improves pedestrian criticality classification performance by up to 50 %. In addition, the proposed criticality prediction framework outperforms state-of-the-art metrics by 275 %, achieving an F1-score of 0.96 and enabling scenario-independent criticality assessment across all object classes. These results demonstrate the strong potential of the proposed approaches to enhance criticality assessment for safety evaluation in automated driving systems.