mlsanityData to 7 Oct 2026

Modeling Human Driving Behavior through Generative Adversarial Imitation Learning

arXiv 2006.06412 ↗First posted 10 Jun 2020cs.AIcited by 43

On the map: Offline RL and imitation learning from data → Imitation and offline RL for driving

Abstract (arXiv)

An open problem in autonomous vehicle safety validation is building reliable models of human driving behavior in simulation. This work presents an approach to learn neural driving policies from real world driving demonstration data. We model human driving as a sequential decision making problem that is characterized by non-linearity and stochasticity, and unknown underlying cost functions. Imitation learning is an approach for generating intelligent behavior when the cost function is unknown or difficult to specify. Building upon work in inverse reinforcement learning (IRL), Generative Adversarial Imitation Learning (GAIL) aims to provide effective imitation even for problems with large or continuous state and action spaces, such as modeling human driving. This article describes the use of GAIL for learning-based driver modeling. Because driver modeling is inherently a multi-agent problem, where the interaction between agents needs to be modeled, this paper describes a parameter-sharing extension of GAIL called PS-GAIL to tackle multi-agent driver modeling. In addition, GAIL is domain agnostic, making it difficult to encode specific knowledge relevant to driving in the learning process. This paper describes Reward Augmented Imitation Learning (RAIL), which modifies the reward signal to provide domain-specific knowledge to the agent. Finally, human demonstrations are dependent upon latent factors that may not be captured by GAIL. This paper describes Burn-InfoGAIL, which allows for disentanglement of latent variability in demonstrations. Imitation learning experiments are performed using NGSIM, a real-world highway driving dataset. Experiments show that these modifications to GAIL can successfully model highway driving behavior, accurately replicating human demonstrations and generating realistic, emergent behavior in the traffic flow arising from the interaction between driving agents.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data. Builds on 31 of its 85 references that are in the map. First posted on arXiv 10 Jun 2020.

Builds on

31 of 85 references.

Closest ancestors · 2

Same direction, newest first

  1. 2019Simulating Emergent Properties of Human Driving Behavior Using Multi-Agent Reward Augmented Imitation Learning14 Mar
  2. 2018Multi-Agent Imitation Learning for Driving Simulation2 Mar

Other references · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2017Improved Training of Wasserstein GANscited by 2,223
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  7. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  8. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  9. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
  10. 2016Learning Multiagent Communication with Backpropagationcited by 355
  11. 2019A Divergence Minimization Perspective on Imitation Learning Methodscited by 188
  12. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 178
17 more
  1. 2019Multiple Futures Predictioncited by 178
  2. 2018Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applicationscited by 122
  3. 2018Multi-Agent Generative Adversarial Imitation Learningcited by 99
  4. 2017Imitating Driver Behavior with Generative Adversarial Networkscited by 92
  5. 2018An LSTM Network for Highway Trajectory Predictioncited by 81
  6. 2019Multi-Agent Adversarial Inverse Reinforcement Learningcited by 60
  7. 2019Wasserstein Adversarial Imitation Learningcited by 59
  8. 2017Probabilistic Vehicle Trajectory Prediction over Occupancy Grid Map via Recurrent Neural Networkcited by 56
  9. 2016Model-Free Imitation Learning with Policy Optimizationcited by 52
  10. 2018Hierarchical Game-Theoretic Planning for Autonomous Vehiclescited by 48
  11. 2018How would surround vehicles move? A Unified Framework for Maneuver Classification and Motion Predictioncited by 46
  12. 2017Lenient Multi-Agent Deep Reinforcement Learningcited by 35
  13. 2019SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewardscited by 28
  14. 2017Simultaneous Policy Learning and Latent State Inference for Imitating Driver Behaviorcited by 16
  15. 2019Interactive Decision Making for Autonomous Vehicles in Dense Trafficcited by 11
  16. 2017Burn-In Demonstrations for Multi-Modal Imitation Learningcited by 9
  17. 2019Decision Making for Autonomous Vehicles at Unsignalized Intersection in Presence of Malicious Vehiclescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 6; 2023: 9; 2024: 12; 2025: 8; 2026: 3.

43 papers.

Sort:

By area:

  1. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  2. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  3. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
  4. 2022CitySim: A Drone-Based Vehicle Trajectory Dataset for Safety Oriented Research and Digital Twinscited by 43
  5. 2024WcDT: World-centric Diffusion Transformer for Traffic Scene Generationcited by 31
  6. 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
  7. 2023SceneDM: Scene-level Multi-agent Trajectory Generation with Consistent Diffusion Modelscited by 24
  8. 2024The Role of Predictive Uncertainty and Diversity in Embodied AI and Robot Learningcited by 11
  9. 2023Residual Q-Learning: Offline and Online Policy Customization without Valuecited by 9
  10. 2025Diverse Controllable Diffusion Policy with Signal Temporal Logiccited by 8
  11. 2022RITA: Boost Driving Simulators with Realistic Interactive Traffic Flowcited by 7
  12. 2025SPACeR: Self-Play Anchoring with Centralized Reference Modelscited by 7
31 more
  1. 2021Grounded Relational Inference: Domain Knowledge Driven Explainable Autonomous Drivingcited by 6
  2. 2021A Hybrid Rule-Based and Data-Driven Approach to Driver Modeling through Particle Filteringcited by 6
  3. 2024Integration of Mixture of Experts and Multimodal Generative AI in Internet of Vehicles: A Surveycited by 6
  4. 2023Curricular Subgoals for Inverse Reinforcement Learningcited by 5
  5. 2025DecompGAIL: Learning Realistic Traffic Behaviors with Decomposed Multi-Agent Generative Adversarial Imitation Learningcited by 5
  6. 2024A Gentle Introduction and Tutorial on Deep Generative Models in Transportation Researchcited by 4
  7. 2022DeepIPC: Deeply Integrated Perception and Control for an Autonomous Vehicle in Real Environmentscited by 3
  8. 2024BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human Racing Gameplaycited by 3
  9. 2024Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Reviewcited by 3
  10. 2024Efficient Driving Behavior Narration and Reasoning on Edge Device Using Large Language Modelscited by 3
  11. 2025HAD-Gen: Human-like and Diverse Driving Behavior Modeling for Controllable Scenario Generationcited by 3
  12. 2021How To Not Drive: Learning Driving Constraints from Demonstrationcited by 2
  13. 2022Fail-Safe Adversarial Generative Imitation Learningcited by 2
  14. 2023Learning An Active Inference Model of Driver Perception and Control: Application to Vehicle Car-Followingcited by 2
  15. 2022Modeling Human Behavior Part I -- Learning and Belief Approachescited by 1
  16. 2023On Learning the Tail Quantiles of Driving Behavior Distributions via Quantile Regression and Flowscited by 1
  17. 2023Do as I can, not as I getcited by 1
  18. 2024LitSim: A Conflict-aware Policy for Long-term Interactive Traffic Simulationcited by 1
  19. 2024Learning for Long-Horizon Planning via Neuro-Symbolic Abductive Imitationcited by 1
  20. 2025Toward Efficient and Robust Behavior Models for Multi-Agent Driving Simulationcited by 1
  21. 2022An Adaptive Human Driver Model for Realistic Race Car Simulationscited by 0
  22. 2023Specification-Guided Data Aggregation for Semantically Aware Imitation Learningcited by 0
  23. 2024MRIC: Model-Based Reinforcement-Imitation Learning with Mixture-of-Codebooks for Autonomous Driving Simulationcited by 0
  24. 2024SoftCTRL: Soft conservative KL-control of Transformer Reinforcement Learning for Autonomous Drivingcited by 0
  25. 2024SIL-RRT*: Learning Sampling Distribution through Self Imitation Learningcited by 0
  26. 2025Trajectory Prediction in Dynamic Object Tracking: A Critical Studycited by 0
  27. 2025Group Effect Enhanced Generative Adversarial Imitation Learning for Individual Travel Behavior Modeling under Incentivescited by 0
  28. 2025BLIP-FusePPO: A Vision-Language Deep Reinforcement Learning Framework for Lane Keeping in Autonomous Vehiclescited by 0
  29. 2026Artificial Intelligence for Modeling and Simulation of Mixed Automated and Human Trafficcited by 0
  30. 2026Improving Crash Frequency Prediction from Simulated Traffic Conflicts Using Machine Learning Based Microsimulationcited by 0
  31. 2026STAGE: STyle-controllable Action GEneration for personalized autonomous drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Generative Adversarial Imitation Learning for End-to-End Autonomous Driving on Urban Environments
  2. 2020Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets
  3. 2024Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priors
  4. 2022SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments
  5. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving
  6. 2019Decision Making for Autonomous Driving via Augmented Adversarial Inverse Reinforcement Learning
  7. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios
  8. 2020On Computation and Generalization of Generative Adversarial Imitation Learning