Investigating Behavior Cloning from Few Demonstrations for Autonomous Driving based on Bird's-Eye View in Simulated Cities

AI summary

Overview

Research area: Imitation learning / behavior cloning for autonomous driving in simulation (Robotics, cs.RO).

Technical level: Intermediate. The paper assumes familiarity with convolutional neural networks, supervised learning loss functions, and basic probability distributions, but the core idea — re-weighting training samples so that rare actions matter more — is explained concretely.

Scope: A study of whether a density-weighted behavior cloning loss, trained on a small set of expert routes with bird's-eye-view inputs, lets a simulated vehicle complete routes and obey traffic lights in both a seen and an unseen CARLA city.

What This Paper Is About

Behavior cloning (BC) teaches a driving policy by supervised learning on expert demonstrations, but plain BC struggles when some expert actions are much rarer than others — exactly the case for stopping at a red light and pulling away on green. The authors investigate whether weighting each training sample by the inverse density of its action lets a BC agent learn those rare-but-critical behaviors from a small number of expert routes. They test this in CARLA using bird's-eye-view (BEV) images instead of raw camera pixels.

Key Contributions

1. A density-weighted behavior cloning loss: a kernel density estimator (KDE) is fitted to the expert action distribution, and each observation-action pair is weighted by the inverse of its estimated density, w_i = 1 / ĝ(a_i). This increases the influence of rare actions such as hard braking at red lights and the short acceleration burst after a light turns green.

2. A BEV-based input representation for the driving policy: six 192×192 channels combining the desired route, drivable area, and lane boundaries, plus three channels encoding the history of traffic-light state at timesteps −16, −9, and −1. These mid-level inputs decouple feature representation from policy learning.

3. A stochastic Beta-distribution policy with a bounded action space of [-1, 1]² (steering and acceleration), where the network outputs the Beta shape parameters α and β for each actuator; braking is implied by negative acceleration rather than being a separate action.

4. An experimental comparison of weighted BC against standard BC on a training city (town01, 8 of 10 routes used for training) and an unseen test city (town02), including a comparison against the expert agent that generated the data.

Main Findings

• Weighted BC matches the expert in the training city. In town01, Weighted BC reached a driving score of 100 ± 0, route completion of 100 ± 0, infraction penalty of 100 ± 0, and 0 ± 0 red light infractions per km. Standard BC reached a driving score of 81 ± 20, route completion 83 ± 18, infraction penalty 97 ± 5, and 0.39 ± 0.68 red light infractions per km.

• The gap is largest in the unseen city. In town02, Weighted BC scored 98 ± 0.03 with route completion 100 ± 0 and infraction penalty 98 ± 0.03, while standard BC scored 34 ± 18 with route completion 53 ± 18 and infraction penalty 66 ± 20.

• Weighted BC eliminated red-light infractions in the test city. In town02, Weighted BC recorded 0 ± 0 red light infractions per km; standard BC recorded 3.08 ± 2.45. The expert recorded 0 ± 0 in both cities.

• The expert set the ceiling. The expert achieved a driving score, route completion, and infraction penalty of 100 ± 0 in both town01 and town02, with 0 ± 0 red light infractions per km, crossing 4.7 ± 1.9 traffic lights in town01 and 9.0 ± 2.16 in town02.

• Learning was faster with weighting. Evaluated every eight training epochs on all ten town01 routes across four random runs per agent, Weighted BC reached the minimum error (zero incomplete routes) sooner than standard BC, which did not reach zero infractions by the end of training.

• Action density is strongly non-uniform. Plotting the logarithm of the KDE density along a town01 route showed the highest density for driving straight with constant acceleration and zero steering (for example, between timesteps 470 and 900), while the least frequent action was the higher, shorter acceleration period after a traffic light switches from red to green. On that route the expert encountered three red traffic lights and made two left turns and two right turns.

• Driving score is a product, not a sum. The CARLA Leaderboard driving score multiplies route completion by an infraction penalty; for instance, running two red lights at a penalty of 0.7 each gives 0.7 × 0.7 = 0.49.

• Scale of the data. The ten expert trajectories were recorded at 10 Hz (10 observation-action pairs per second), producing 21,287 labeled samples (30 GB uncompressed), corresponding to roughly 36 minutes or 8 km of driving. The shortest route had 1480 samples and the average route 2129 samples, i.e. about 2.5 and 3.5 minutes of simulated driving respectively.

Methodology in Plain English

The researchers first built an expert dataset rather than collecting human driving. A deterministic agent in the CARLA simulator follows a dense point trajectory using a classic PID controller, brakes when it detects a red light, and accelerates when the light turns green. The dense trajectory is used only to generate labels; the learning agent itself sees only a sparse trajectory (points every 50 meters, or when the vehicle is about to change movement) for general guidance. The data comes from the town01 environment of the CARLA Leaderboard platform and ten predefined routes.

For input, the agent does not see camera pixels. Instead it receives a mid-level bird's-eye view: a top-down 192×192 image with six channels — route, drivable area, lane boundaries, and three snapshots of the traffic light state from earlier timesteps. The vehicle's current speed and its last executed acceleration and steering values are fed in separately.

The policy network processes the BEV image with a CNN into a 256-dimensional embedding, processes the scalar state variables with a multi-layer perceptron into another 256-dimensional vector, concatenates the two, and passes the result through a final MLP. The output is not a single action value but the α and β parameters of a Beta distribution for acceleration and for steering, which lets the model represent a range of plausible actions within the bounded interval.

Training uses the negative log-likelihood of the Beta distribution. The key twist is that each sample's contribution is multiplied by a weight: a Gaussian kernel density estimate of the expert's actions is computed, and the weight is the inverse of that density, so rarely-taken actions contribute more to the gradient. Two agents were trained identically except for this weighting, using 8 routes from town01. Every eight epochs both were evaluated on all ten town01 routes, with four random runs per agent, and their final policies were also run in the unseen town02 city.

Why This Matters

Impact on research. The paper shows that a simple, cheap change to the loss function — re-weighting by inverse action density — can recover behavior that a standard behavior cloning objective misses when demonstrations are few and the action distribution is skewed. This is relevant to anyone working on imitation learning with imbalanced demonstration data, not only driving.

Real-world applications:

• Autonomous vehicle perception-and-control stacks that must reliably handle rare safety-critical events such as stopping for red lights.
• Robot learning from demonstration in industrial or domestic settings, where a human demonstrator will naturally spend most time doing the easy, common motions.
• Sim-to-real pipelines where collecting expert data is expensive and only a small number of demonstration runs is available.
• Any supervised control problem with a bounded action space that can be modeled with a Beta distribution.

Industry relevance. Fleet data is dominated by uneventful driving, so the rare braking and stopping events that matter most for safety are exactly the ones a naive loss will under-learn. The paper's approach points to a low-cost way of getting more safety-relevant behavior out of existing demonstration logs, and the demonstrated generalization from town01 to town02 is a signal that the learned policy is not simply memorizing one map.

Future Directions

• Extending the approach to cities containing pedestrians and other vehicles, which the current town01/town02 setup with the described evaluation does not include.
• Applying more advanced behavior cloning methods such as implicit behavioral cloning to handle multimodal action distributions — cases where several different actions are reasonable in the same situation.
• Investigating how the method scales beyond the small demonstration sets used here (8 training routes, 21,287 labeled samples), since the paper frames scarcity of demonstrations as the motivating constraint.
• Understanding more precisely why the weighted loss transfers better to an unseen city, given that standard BC's driving score drops from 81 in town01 to 34 in town02 while Weighted BC drops only from 100 to 98.

Target Audience

Researchers and graduate students working on imitation learning, behavior cloning, and end-to-end autonomous driving; practitioners building driving policies in simulators such as CARLA; and roboticists dealing with imbalanced or scarce demonstration data. The paper is readable by someone with a general machine learning background, though the Beta-distribution policy and KDE weighting are most useful to readers already comfortable with probabilistic loss functions.

Authors’ abstract

This paper investigates the use of Behavior Cloning (BC) for autonomous driving from a bird's-eye view (BEV) perspective in simulated urban environments. BC uses supervised learning to mimic expert driving behaviors. Previous works have applied BC in the CARLA simulator but did not fully address the challenges of traffic light compliance. Our approach enhances BC by integrating a kernel density estimator to adjust training sample weights based on action density, thereby improving the learning of rare but critical actions such as stopping at red lights and accelerating at green lights, specially in scenarios of scarce number of expert demonstrations. Using BEV inputs, which provide an abstract top-down view of the driving environment, our method simplifies the policy learning process. The trained convolutional neural network (CNN) outputs steering and acceleration actions based on these BEV inputs and additional state variables. Experimental results in the CARLA simulator demonstrate that our weighted BC method significantly improves driving performance, achieving higher route completion compared to standard BC. This weighted approach proved to be crucial in learning correct driving behaviors, particularly in test environments not encountered during training, highlighting its potential for enhancing autonomous vehicle navigation.