mlsanityData to 7 Oct 2026

Multi-Agent Imitation Learning for Driving Simulation

arXiv 1803.01044 ↗First posted 2 Mar 2018cs.AIcited by 49

On the map: Offline RL and imitation learning from data → Imitation and offline RL for driving

Abstract (arXiv)

Simulation is an appealing option for validating the safety of autonomous vehicles. Generative Adversarial Imitation Learning (GAIL) has recently been shown to learn representative human driver models. These human driver models were learned through training in single-agent environments, but they have difficulty in generalizing to multi-agent driving scenarios. We argue these difficulties arise because observations at training and test time are sampled from different distributions. This difference makes such models unsuitable for the simulation of driving scenes, where multiple agents must interact realistically over long time horizons. We extend GAIL to address these shortcomings through a parameter-sharing approach grounded in curriculum learning. Compared with single-agent GAIL policies, policies generated by our PS-GAIL method prove superior at interacting stably in a multi-agent setting and capturing the emergent behavior of human drivers.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data. Builds on 8 of its 30 references that are in the map. First posted on arXiv 2 Mar 2018.

Builds on

8 of 30 references.

References · 8

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2017Improved Training of Wasserstein GANscited by 2,223
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
  6. 2017Adversarial Learning for Neural Dialogue Generationcited by 103
  7. 2017Imitating Driver Behavior with Generative Adversarial Networkscited by 92
  8. 2017Robust Imitation of Diverse Behaviorscited by 85

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 2; 2020: 1; 2021: 12; 2022: 7; 2023: 6; 2024: 9; 2025: 7; 2026: 4.

49 papers.

Sort:

By area:

  1. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  2. 2023The Waymo Open Sim Agents Challengecited by 86
  3. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learningcited by 59
  4. 2021Imitation Learning: Progress, Taxonomies and Challengescited by 56
  5. 2020Modeling Human Driving Behavior through Generative Adversarial Imitation Learningcited by 43
  6. 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
  7. 2024CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learningcited by 26
  8. 2018Learning from Demonstration in the Wildcited by 25
  9. 2019Simulating Emergent Properties of Human Driving Behavior Using Multi-Agent Reward Augmented Imitation Learningcited by 24
  10. 2022TrajGen: Generating Realistic and Diverse Trajectories with Reactive and Feasible Agent Behaviors for Autonomous Drivingcited by 23
  11. 2021Congestion-aware Multi-agent Trajectory Prediction for Collision Avoidancecited by 21
  12. 2024MAPF-GPT: Imitation Learning for Multi-Agent Pathfinding at Scalecited by 17
37 more
  1. 2021Multi-Agent Imitation Learning with Copulascited by 13
  2. 2024Multi-Agent Imitation Learning: Value is Easy, Regret is Hardcited by 11
  3. 2024Learning to Drive via Asymmetric Self-Playcited by 11
  4. 2022Leveraging Smooth Attention Prior for Multi-Agent Trajectory Predictioncited by 9
  5. 2021Objective-aware Traffic Simulation via Inverse Reinforcement Learningcited by 8
  6. 2019Independent Generative Adversarial Self-Imitation Learning in Cooperative Multiagent Systemscited by 7
  7. 2021Reactive and Safe Road User Simulations using Neural Barrier Certificatescited by 7
  8. 2022RITA: Boost Driving Simulators with Realistic Interactive Traffic Flowcited by 7
  9. 2025DriveGen: Towards Infinite Diverse Traffic Scenarios with Large Modelscited by 7
  10. 2021Grounded Relational Inference: Domain Knowledge Driven Explainable Autonomous Drivingcited by 6
  11. 2021Learning to Simulate on Sparse Trajectory Datacited by 6
  12. 2021A Hybrid Rule-Based and Data-Driven Approach to Driver Modeling through Particle Filteringcited by 6
  13. 2025MIMIC-D: Multi-modal Imitation for MultI-agent Coordination with Decentralized Diffusion Policiescited by 6
  14. 2025DecompGAIL: Learning Realistic Traffic Behaviors with Decomposed Multi-Agent Generative Adversarial Imitation Learningcited by 5
  15. 2022Exploring the trade off between human driving imitation and safety for traffic simulationcited by 3
  16. 2024Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Reviewcited by 3
  17. 2024Enabling Multi-Robot Collaboration from Single-Human Guidancecited by 3
  18. 2021How To Not Drive: Learning Driving Constraints from Demonstrationcited by 2
  19. 2022Semi-Supervised Imitation Learning of Team Policies from Suboptimal Demonstrationscited by 2
  20. 2023ConBaT: Control Barrier Transformer for Safe Policy Learningcited by 2
  21. 2023Inverse Factorized Q-Learning for Cooperative Multi-agent Imitation Learningcited by 2
  22. 2024Conditional Latent ODEs for Motion Prediction in Autonomous Drivingcited by 2
  23. 2025Hierarchical Imitation Learning of Team Behavior from Heterogeneous Demonstrationscited by 2
  24. 2021Spatio-Temporal Look-Ahead Trajectory Prediction using Memory Neural Networkcited by 1
  25. 2022Teamwork under extreme uncertainty: AI for Pokemon ranks 33rd in the worldcited by 1
  26. 2023Toward a Surgeon-in-the-Loop Ophthalmic Robotic Apprentice using Reinforcement and Imitation Learningcited by 1
  27. 2025Curriculum Imitation Learning of Distributed Multi-Robot Policiescited by 1
  28. 2025R2BC: Multi-Agent Imitation Learning from Single-Agent Demonstrationscited by 1
  29. 2022A New Approach to Training Multiple Cooperative Agents for Autonomous Drivingcited by 0
  30. 2023Human-Like Autonomous Driving on Dense Trafficcited by 0
  31. 2024Rethinking Adversarial Inverse Reinforcement Learning: Policy Imitation, Transferable Reward Recovery and Algebraic Equilibrium Proofcited by 0
  32. 2024On Reward Transferability in Adversarial Inverse Reinforcement Learning: Insights from Random Matrix Theorycited by 0
  33. 2025On Generalization and Distributional Update for Mimicking Observations with Adequate Explorationcited by 0
  34. 2026Matching Multiple Experts: On the Exploitability of Multi-Agent Imitation Learningcited by 0
  35. 2026MARL-GPT: Foundation Model for Multi-Agent Reinforcement Learningcited by 0
  36. 2026Artificial Intelligence for Modeling and Simulation of Mixed Automated and Human Trafficcited by 0
  37. 2026Improving Crash Frequency Prediction from Simulated Traffic Conflicts Using Machine Learning Based Microsimulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Generative Adversarial Imitation Learning for End-to-End Autonomous Driving on Urban Environments
  2. 2020Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets
  3. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving
  4. 2024Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priors
  5. 2022SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments
  6. 2024Human-compatible driving partners through data-regularized self-play reinforcement learning
  7. 2022Learning from Demonstrations of Critical Driving Behaviours Using Driver's Risk Field
  8. 2021Learning Interactive Driving Policies via Data-driven Simulation