DecompGAIL: Learning Realistic Traffic Behaviors with Decomposed Multi-Agent Generative Adversarial Imitation Learning

AI summary

Overview

• Research area: Multi-agent imitation learning for realistic traffic simulation, sitting at the intersection of inverse reinforcement learning (IRL), generative adversarial imitation learning (GAIL), and autonomous driving simulation benchmarks.
• Technical level: Advanced. The paper assumes familiarity with Markov games, GAIL minimax objectives, PPO, GAE, and Transformer attention architectures (MHSA/MHCA, relative positional encoding).
• Scope: The paper diagnoses a specific instability in decentralized multi-agent GAIL, proposes a decomposed discriminator plus a social PPO objective to fix it, and evaluates the resulting method (DecompGAIL) on the WOMD Sim Agents 2025 (WOSAC) benchmark.

What This Paper Is About

Traffic simulators that drive realistic multi-agent behavior are needed to test autonomous vehicles safely, but existing imitation learning methods struggle: behavior cloning suffers from covariate shift (compounding errors once the policy drifts off expert states), and GAIL becomes unstable in multi-agent settings. The authors identify a root cause they call "irrelevant interaction misguidance" — a decentralized discriminator gives an ego vehicle a low realism score because its neighbors behaved unrealistically, even when the ego itself drove realistically. The goal is a discriminator design whose reward signal depends only on what the ego actually controls, yielding stable training and higher realism.

Key Contributions

1. Identification of irrelevant interaction misguidance. The authors show conceptually and empirically that a standard parameter-sharing GAIL (PS-GAIL) discriminator implicitly decomposes its score into four terms — ego–map (scene) realism, ego–neighbor (interaction) realism, neighbor–map/neighbor–neighbor signals, and higher-order correlations. The third term is only weakly coupled to the ego's action yet grows roughly quadratically with neighborhood size, making rewards noisier and less informative as the number of neighbors increases.

2. A decomposed discriminator (DecompGAIL). Instead of a monolithic discriminator over the ego's local observation, the method computes scene realism from ego–map inputs via an MLP over map-attention features, and interaction realism from pairwise ego–neighbor inputs via an MLP over concatenated temporal features and relative positional encodings. Neighbor–neighbor and neighbor–map signals are excluded by design. Pairwise interaction terms are combined with distance-decayed weights w_ij = α exp(−d(i,j)/β).

3. A social PPO objective. The per-agent adversarial reward is augmented with distance-weighted rewards from neighboring agents (λ_ij weights analogous to w_ij), so agents improve their own realism without degrading that of nearby agents. Training follows IPPO with social rewards, GAE for advantages, and a combined PPO plus behavior cloning loss.

4. State-of-the-art results on WOMD Sim Agents 2025. Integrated into a lightweight SMART-based backbone with behavior cloning pretraining, DecompGAIL achieves the best realism meta-metric among the compared baselines on the WOSAC 2025 leaderboard test split.

Main Findings

• Stability scales better than PS-GAIL. In the training-stability comparison — PS-GAIL with 5, 10, and all available nearest neighbors within 60 m versus DecompGAIL with all neighbors within the same range — PS-GAIL shows increasing variance and a lower mean score as neighbor count grows. DecompGAIL maintains low variance with a mean near the expected equilibrium of 0.5, and its simulation performance improves steadily throughout training (realism meta-metric evaluated on the 2% validation split).
• Best realism meta-metric on the WOSAC 2025 leaderboard test split. SMART-tiny-DecompGAIL scores 0.7864, ahead of SMART-R1 (0.7858), SMART-tiny-RLFTSim (0.7857), TrajTok (0.7852), SMART-tiny-CLSFT (0.7846), UniMM (0.7829), SMART-tiny (0.7814), LLM2AD (0.7779), and InfGen (0.7731).
• Strong interactive metrics, competitive elsewhere, higher minADE. DecompGAIL records 0.8152 on the interactive metric (the highest among the listed models), 0.4919 kinematic, and 0.9176 map-based. Its minADE of 1.4209 is higher than most baselines (e.g., SMART-R1 at 1.2885, UniMM at 1.2949), which the authors attribute to prioritizing feature-distribution matching between expert and policy trajectories over distance-based similarity.
• Ablation: every component contributes. On the 2% validation split, full DecompGAIL reaches a meta-metric of 0.7889, kinematic 0.5148, interactive 0.8283, map-based 0.8948, and collision likelihood 0.9837. Removing DecompGAIL fine-tuning entirely drops the meta-metric to 0.7836 and collision likelihood to 0.9667; removing scene realism drops map-based to 0.8795; removing interaction realism drops interactive to 0.8132 and collision likelihood to 0.9573; replacing distance-weighted interaction aggregation with uniform averaging gives 0.7819; removing neighborhood rewards gives 0.7871; using uniform averaging for neighborhood rewards gives 0.7882.
• Gradient penalties do not solve the problem, and can hurt. Applying WGAN-GP and R1/R2 regularization (penalty weight fixed to 1) to PS-GAIL raises its meta-metric from 0.7674 to 0.7723 (WGAN-GP), 0.7776 (R1), and 0.7706 (R2), but all remain well below DecompGAIL's 0.7889. Adding the same penalties to DecompGAIL lowers realism below the unpenalized 0.7889 (0.7865 for WGAN-GP, 0.7861 for R1, 0.7840 for R2), even though kinematic scores improve slightly — the authors argue collision and off-road realism depend on non-smooth decision boundaries that gradient penalties suppress.
• Hyperparameter sensitivity. Sweeping interaction weights over α ∈ {2.5, 5, 10, 20} and β ∈ {1, 2.5, 5, 10} (social rewards disabled), performance is more sensitive to the decay range β than to the scale α, and both very small and very large β degrade realism. With interaction weights fixed at the optimum (α = 10, β = 2.5) and social-reward parameters swept over α ∈ {0.5, 1, 5, 10} and β ∈ {1, 2.5, 5, 10}, social reward parameters have weaker influence, though very large values make neighborhood weights nearly uniform and increase reward noise.
• Freezing the map encoder helps slightly. Fine-tuning the map encoder instead of freezing it yields slightly lower training stability and marginally lower realism.

Methodology in Plain English

The authors start from a standard traffic model called SMART, which is a factorized Transformer that predicts the next "motion token" for every agent in a scene, conditioned on map features and all agents' past motion. They first train this policy with behavior cloning — straightforward supervised imitation of expert trajectories — for 32 epochs in about 44 hours.

Then they switch to adversarial fine-tuning, which is where the novelty lies. A standard GAIL discriminator looks at an agent's local view (the agent's own history plus its neighbors' histories and the map) and outputs a realism score. The problem: if a neighbor drives badly, the discriminator penalizes the ego agent too. The fix is to break the score into pieces. One head judges "scene realism" using only the ego's own trajectory and the map. A second head judges "interaction realism" separately for each ego–neighbor pair, using the concatenated temporal features of the two agents and their relative positional encoding. Neighbor–neighbor and neighbor–map information is never fed into the score, so it cannot mislead the reward.

The pairwise interaction scores are combined with weights that decay with distance, so nearby agents matter more than far ones. Each agent's reward is the sum of its scene score and its weighted interaction scores, expressed as negative log-likelihoods of the discriminator outputs. To keep the population realistic as a whole, each agent's reward is further augmented with distance-weighted rewards of its neighbors — this is the "social" component. The combined social rewards are optimized with independent PPO using GAE, with a behavior cloning loss mixed in for stability. Fine-tuning takes 2 additional epochs, about 10 hours, on 8×H800 80GB GPUs with a total batch size of 80; the map encoder is frozen and shared between the policy and discriminator.

Why This Matters

The paper matters because it moves beyond treating multi-agent GAIL instability as something to be papered over with parameter sharing, curriculum learning, or hand-crafted rewards, and instead names a concrete structural cause — the discriminator attending to interactions the ego agent does not control — and removes it architecturally. The ablation and gradient-penalty experiments support that this is a better-targeted fix than regularization, which the authors show can actively hurt non-smooth safety metrics like collision and off-road likelihood.

Real-world applications noted or implied in the paper:

• Autonomous driving system validation: high-fidelity simulators allow safe, controlled evaluation of driving policies before on-road deployment.
• Stress-testing safety-critical scenarios: realistic simulated traffic lets engineers reproduce rare and dangerous situations repeatedly.
• Urban mobility planning: planners can study traffic interventions and their effects on a realistic multi-agent population.
• Scenario generation for training and testing: collision-free, on-road multi-agent rollouts over the full 8-second horizon provide abundant, realistic interactive scenarios.

Industry relevance: the fourth author is affiliated with Desay SV Automotive, an automotive supplier, and the method is built on SMART, a widely used open traffic model already adopted by several prior works. That makes the approach relatively easy to drop into existing learning-based simulation stacks — it changes the discriminator and reward design rather than requiring a new backbone.

Future Directions

• Closing the minADE gap. DecompGAIL's minADE of 1.4209 on the leaderboard test split is higher than most baselines, which the authors attribute to matching feature distributions rather than distances. Reconciling distribution-matching realism with low displacement error is left open.
• Reducing compute cost. Pretraining takes about 44 hours and fine-tuning about 10 hours on 8×H800 80GB GPUs, and fine-tuning runs for only 2 epochs. Whether longer or cheaper fine-tuning changes the outcome is not reported.
• Extending beyond one benchmark. All reported results are on WOMD and the WOSAC 2025 challenge; generalization to other datasets or simulator settings is not evaluated.
• Refining the decomposition. The decomposed discriminator deliberately drops higher-order terms (φ₃ and φ₄ in the paper's decomposition). Whether any of those signals carry useful information that could be retained in a controlled way, and how the social reward weights λ_ij interact with the interaction weights w_ij at larger neighborhood sizes, remain open questions.

Target Audience

This paper is best suited to researchers and graduate students working on imitation learning, inverse reinforcement learning, or multi-agent reinforcement learning, particularly those applying these methods to autonomous driving and traffic simulation. It is also relevant to practitioners building or validating learning-based traffic simulators who need stable adversarial fine-tuning, and to benchmark-oriented engineers tracking the WOMD Sim Agents 2025 leaderboard. Readers without a background in Markov games, GAIL objectives, or PPO will find the mathematical sections demanding.

Authors’ abstract

Realistic traffic simulation is critical for the development of autonomous driving systems and urban mobility planning, yet existing imitation learning approaches often fail to model realistic traffic behaviors. Behavior cloning suffers from covariate shift, while Generative Adversarial Imitation Learning (GAIL) is notoriously unstable in multi-agent settings. We identify a key source of this instability: irrelevant interaction misguidance, where a discriminator penalizes an ego vehicle's realistic behavior due to unrealistic interactions among its neighbors. To address this, we propose Decomposed Multi-agent GAIL (DecompGAIL), which explicitly decomposes realism into ego-map and ego-neighbor components, filtering out misleading neighbor: neighbor and neighbor: map interactions. We further introduce a social PPO objective that augments ego rewards with distance-weighted neighborhood rewards, encouraging overall realism across agents. Integrated into a lightweight SMART-based backbone, DecompGAIL achieves state-of-the-art performance on the WOMD Sim Agents 2025 benchmark.