STAGE: STyle-controllable Action GEneration for personalized autonomous driving

AI summary

Overview

Research area: Robotics and autonomous driving — specifically imitation learning, driving-style modeling, and human-in-the-loop vehicle control.

Technical level: Advanced. The paper assumes familiarity with imitation learning, Transformer encoder-decoder architectures, Variational Autoencoders (VAEs), Kullback-Leibler (KL) divergence, and preference learning (the Bradley-Terry style formulation used here).

Scope: The paper introduces STAGE (STyle-controllable Action GEneration), a Transformer-based imitation learning system that learns a continuous, monotonic driving-style value from pairwise preferences and lets a driver dial that value at inference time to control how aggressively the vehicle plans trajectories and issues steering and throttle commands.

What This Paper Is About

Driving style — how aggressive or cautious a person drives — is shaped by individual habits, but most autonomous driving systems offer no way to match it. Existing style-recognition methods either sort drivers into fixed classes (aggressive, moderate, conservative) or learn entangled latent spaces with no intuitive "knob" tied to a specific driving characteristic. STAGE's goal is to learn a single continuous number representing driving aggressiveness that a driver can set directly, so the generated trajectories and control signals feel like the driver's own habits.

Key Contributions

1. An action modality module built on an encoder-decoder architecture that takes state and action as input and separates them into two parts: a style value representing driving aggressiveness and a VAE latent capturing the remaining policy patterns.

2. A preference learning method for driving style that produces continuous, consistent style assessments across scenarios, overcoming the discrete-only outputs of prior approaches. The authors also designed aggressiveness-scoring rules that automate the preference comparisons between action pairs, reducing the manual annotation effort preference learning normally requires.

3. A style-conditioned action generation method that combines policy imitation with a user intervention mechanism, letting a user input a desired style value during model execution so the policy generates outputs consistent with that style.

4. An experimental demonstration across three evaluation sets — style value behavior, style value alignment, and safety — comparing STAGE against DETR-based behavioral cloning (BC), GAIL, CVAE, CVAE with discrete style, and a preference-style-only variant of STAGE.

Main Findings

• Style value controls behavior qualitatively as expected. In representative simulator scenarios (left turn at an intersection, right turn at an intersection, and overtaking on a straight road), larger style values produced planned trajectories with higher future speeds. In the unprotected left-turn case, the style value directly influenced the length of the planned trajectory. On straight roads, low style values kept the vehicle at the same speed as other vehicles or waiting behind them, while high style values generated overtaking trajectories.

• Turning behavior can shift with style value. In the right-turn scenarios, different style values sometimes changed which lane the vehicle turned into; the authors attribute this to the model accounting for motion capability constraints allowed by the vehicle's speed. Because nearby vehicles were present, generated trajectories stayed relatively conservative to avoid collisions.

• STAGE and its preference-style variant are the only compared methods that achieve both style controllability and style continuity simultaneously. The paper states that BC (DETR), GAIL, CVAE, and CVAE+Discrete Style fail to satisfy both criteria at once.

• Style alignment is better with a continuous style representation. When human style value sequences (computed by STAGE's style value module) were fed into both STAGE and the discrete-style CVAE model, the Spearman correlation coefficient R² of STAGE was closer to 1, indicating superior reproduction of human intent style values. Discrete styles also showed short-term abrupt changes and instability. Note that in this comparison the STAGE style values were normalized to [-1, 1] while the discrete three-category styles (conservative, moderate, aggressive) were mapped to -1, 0, 1.

• Average Completion Rate (mean ± standard deviation of the percentage of mileage safely completed across 20 randomly generated driving scenarios in simulation):

  Method	Average Completion Rate / %
  BC (based on DETR)	51.0 (±22.8)
  GAIL	42.6 (±28.8)
  CVAE	83.4 (±15.4)
  CVAE+Discrete Style	78.6 (±28.6)
  BC+Preference Style	62.1 (±25.1)
  STAGE (ours)	92.1 (±20.1)
  Expert Driver	100.0

• Action modality learning improves safety. BC and GAIL scored lowest on completion rate, attributed to their lack of explicit action modality learning. Preference-based BC captured some action modalities through aggressiveness scoring but missed style-independent action modalities, yielding suboptimal safety. The CVAE-based methods (CVAE, CVAE+Discrete Style, STAGE) produced the best results.

• Control-signal statistics indicate improved comfort. GAIL, BC, and CVAE — which lack driving style learning — showed a higher proportion of heavy throttle behavior, and BC and CVAE showed the most frequent braking. STAGE and BC+Preference Style exhibited fewer braking actions and less extreme acceleration, typically using light throttle.

• Style values can produce inconsistent perceived aggressiveness across scenarios. The same style value in different road scenarios may be perceived by the same driver as different levels of aggressiveness, potentially requiring frequent adjustment.

Methodology in Plain English

The system has two stages. First, a style-learning stage: the researchers avoid expensive human labeling by writing an aggressiveness scoring rule. Speed and throttle amplitude are proportional to the aggressiveness score; when the nearest other vehicle is within 20 meters, two additional terms apply — one inversely proportional to the distance to the nearest vehicle (closer driving is more aggressive), and one proportional to deviation from the lane line (overtaking is aggressive). This rule orders randomly selected action pairs, and a preference loss (a negative log-likelihood over the probability that one state-action pair is more aggressive than another) trains a network to output a continuous style value consistent with that ordering. Style values were scaled to the range [-1, 1], where 0 is regular driving, negative is conservative, and positive is aggressive.

Second, a generation stage: a Transformer encoder-decoder (based on the DETR architecture) fuses multimodal input — map images encoded with ResNet into token embeddings, and vector information such as radar scan points and the historical trajectories of the ego vehicle and other vehicles mapped into tokens via an MLP. The action modality is embedded as its own token. The decoder outputs the ego vehicle's planning trajectory plus steering, throttle, and brake commands, supervised by an MAE loss against dataset actions. The overall loss combines the MAE loss, the preference loss, and a VAE KL divergence term, weighted by coefficients λ₁ and λ₂.

At inference, the action modality encoder is not needed. The VAE latent is set to zero to retrieve the most well-trained actions for stability, and the user supplies the style value manually — a larger input yields more aggressive behavior. Data was collected in the MetaDrive simulator using a Logitech G923 steering wheel and pedal set, with the style value entered at test time through a slider implemented with the PySimpleGUI library. MetaDrive uses the Intelligent Driver Model (IDM) for other vehicles, so all dataset trajectories are generated in a closed loop.

Why This Matters

For research, STAGE shows that preference learning can impose structure on a driving-style latent dimension, giving a monotonic, semantically aligned spectrum instead of the entangled latent spaces of standard VAE or GAN approaches — and it does so without expensive human annotation by substituting automated aggressiveness scoring rules. It also offers a concrete alternative to natural-language prompting in Vision-Language-Action (VLA) driving models, which the authors argue suffers from ambiguity and granularity problems ("drive a bit faster" lacks a precise definition).

Real-world applications:

• Personalized advanced driver assistance and autonomous driving. A driver could set a style value to make the vehicle behave more like their own habits, which the paper argues could increase trust and reduce the likelihood of drivers taking over control.
• Human-in-the-loop shared control. The style value acts as a low-attention-cost intervention channel, letting a human shift the vehicle's aggressiveness without taking the wheel.
• Driver monitoring and style characterization. The learned style value module can score a given planning trajectory and control signal in a given state, quantifying aggressiveness — useful for evaluating other drivers or assessing policy outputs.
• Comfort-oriented tuning of control policies. Fewer braking events and less extreme acceleration, observed for STAGE, translate into a smoother ride for passengers.

Industry relevance: Level 2 driving assistance systems are increasingly deployed, and the paper argues that machine driving behavior strongly influences the driver's psychological and behavioral state. A single continuous style parameter is a simpler interface for automakers to expose than a full latent-space editor or a natural-language channel.

Future Directions

• Cross-scenario style consistency. The same style value can be perceived as different aggressiveness levels in different road scenarios, so drivers may have to readjust the value frequently. The authors suggest designing more comprehensive aggression scoring mechanisms that account for diverse driving scenarios.
• Multidimensional style values. The style value V(x, a) can theoretically be extended to a vector covering richer driving characteristics, but learning such a vector requires a larger data scale to accommodate combinations across dimensions — a condition the authors state they currently do not meet.
• More efficient style representations. The paper proposes exploring alternatives such as natural language as a style representation.
• Integration with VLA models. The authors suggest natural-language style control could integrate better with widely used VLA methods, thereby enhancing the safety of model-driven driving.

Target Audience

Researchers and graduate students working on imitation learning, end-to-end autonomous driving, and human factors in driving automation; engineers building personalized driver assistance or style-controllable planning systems; and anyone interested in preference-based reward or value modeling applied to continuous behavior control. Readers need a working knowledge of Transformers, VAEs, and imitation learning terminology to follow the method sections. The paper reports that code is available at github.com/CarlDegio/STAGE.

Authors’ abstract

Driving style refers to the behavioral preferences that drivers maintain during driving, shaped by their diverse experiences, habits, and needs, and is typically reflected in varying levels of aggressiveness. If humans choose to use autonomous driving systems, they would expect the driving style of the systems to closely resemble their own habit. However, this is challenging for current industrial autonomous driving systems. To address this, we developed a style controllable action generation method, STAGE, for driving tasks. Its training process is based on imitation learning, incorporating both style value and latent value action modality encoding. Preference learning is then used to identify the user's driving style as a continuous, monotonic style value. And to reduce the cost of human involvement in the preference training process, we also developed a set of rules to compare driving style in data pairs. Then, during inference, the user inputs the style value to control the generated action patterns, dynamically meeting the user's expectations. Using the STAGE method, we verified that the style-controlled action generation results in several typical road scenarios significantly align with human expectations. Furthermore, through comparisons between the STAGE method and various other approaches, we reveal the unique functionalities of STAGE, including its style controllability, style continuity, driving style alignment capability and driving safety. The code for this work is available at: https://github.com/CarlDegio/STAGE