# Modeling Human Driving Behavior through Generative Adversarial Imitation Learning

arXiv: https://arxiv.org/abs/2006.06412 · HTML: https://mlsanity.com/paper/2006.06412/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 31 of its 85 references that are in the map.
- First posted on arXiv 10 Jun 2020.

## Placed in

- Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/)

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [Learning Multiagent Communication with Backpropagation](https://mlsanity.com/paper/1605.07736/) (2016)
- [A Divergence Minimization Perspective on Imitation Learning Methods](https://mlsanity.com/paper/1911.02256/) (2019)
- [PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settings](https://mlsanity.com/paper/1905.01296/) (2019)
- [Multiple Futures Prediction](https://mlsanity.com/paper/1911.00997/) (2019)
- [Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications](https://mlsanity.com/paper/1812.11794/) (2018)
- [Multi-Agent Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1807.09936/) (2018)
- [Imitating Driver Behavior with Generative Adversarial Networks](https://mlsanity.com/paper/1701.06699/) (2017)
- [An LSTM Network for Highway Trajectory Prediction](https://mlsanity.com/paper/1801.07962/) (2018)
- [Wasserstein Adversarial Imitation Learning](https://mlsanity.com/paper/1906.08113/) (2019)
- [Multi-Agent Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1907.13220/) (2019)
- [Probabilistic Vehicle Trajectory Prediction over Occupancy Grid Map via Recurrent Neural Network](https://mlsanity.com/paper/1704.07049/) (2017)
- [Model-Free Imitation Learning with Policy Optimization](https://mlsanity.com/paper/1605.08478/) (2016)
- [Multi-Agent Imitation Learning for Driving Simulation](https://mlsanity.com/paper/1803.01044/) (2018)
- [Hierarchical Game-Theoretic Planning for Autonomous Vehicles](https://mlsanity.com/paper/1810.05766/) (2018)
- [How would surround vehicles move? A Unified Framework for Maneuver Classification and Motion Prediction](https://mlsanity.com/paper/1801.06523/) (2018)
- [Lenient Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1707.04402/) (2017)
- [SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards](https://mlsanity.com/paper/1905.11108/) (2019)
- [Simulating Emergent Properties of Human Driving Behavior Using Multi-Agent Reward Augmented Imitation Learning](https://mlsanity.com/paper/1903.05766/) (2019)
- [Simultaneous Policy Learning and Latent State Inference for Imitating Driver Behavior](https://mlsanity.com/paper/1704.05566/) (2017)
- [Interactive Decision Making for Autonomous Vehicles in Dense Traffic](https://mlsanity.com/paper/1909.12914/) (2019)
- [Burn-In Demonstrations for Multi-Modal Imitation Learning](https://mlsanity.com/paper/1710.05090/) (2017)

## Built on by (most cited first, 30 of 45)

- [TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors](https://mlsanity.com/paper/2101.06557/) (2021)
- [A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges](https://mlsanity.com/paper/2309.02473/) (2023)
- [Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients](https://mlsanity.com/paper/2109.13333/) (2021)
- [CitySim: A Drone-Based Vehicle Trajectory Dataset for Safety Oriented Research and Digital Twins](https://mlsanity.com/paper/2208.11036/) (2022)
- [WcDT: World-centric Diffusion Transformer for Traffic Scene Generation](https://mlsanity.com/paper/2404.02082/) (2024)
- [Data-driven Traffic Simulation: A Comprehensive Review](https://mlsanity.com/paper/2310.15975/) (2023)
- [SceneDM: Scene-level Multi-agent Trajectory Generation with Consistent Diffusion Models](https://mlsanity.com/paper/2311.15736/) (2023)
- [The Role of Predictive Uncertainty and Diversity in Embodied AI and Robot Learning](https://mlsanity.com/paper/2405.03164/) (2024)
- [Residual Q-Learning: Offline and Online Policy Customization without Value](https://mlsanity.com/paper/2306.09526/) (2023)
- [Diverse Controllable Diffusion Policy with Signal Temporal Logic](https://mlsanity.com/paper/2503.02924/) (2025)
- [RITA: Boost Driving Simulators with Realistic Interactive Traffic Flow](https://mlsanity.com/paper/2211.03408/) (2022)
- [CARL: Congestion-Aware Reinforcement Learning for Imitation-based Perturbations in Mixed Traffic Control](https://mlsanity.com/paper/2404.00796/) (2024)
- [SPACeR: Self-Play Anchoring with Centralized Reference Models](https://mlsanity.com/paper/2510.18060/) (2025)
- [Grounded Relational Inference: Domain Knowledge Driven Explainable Autonomous Driving](https://mlsanity.com/paper/2102.11905/) (2021)
- [A Hybrid Rule-Based and Data-Driven Approach to Driver Modeling through Particle Filtering](https://mlsanity.com/paper/2108.12820/) (2021)
- [Integration of Mixture of Experts and Multimodal Generative AI in Internet of Vehicles: A Survey](https://mlsanity.com/paper/2404.16356/) (2024)
- [Curricular Subgoals for Inverse Reinforcement Learning](https://mlsanity.com/paper/2306.08232/) (2023)
- [BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human Racing Gameplay](https://mlsanity.com/paper/2402.14194/) (2024)
- [DecompGAIL: Learning Realistic Traffic Behaviors with Decomposed Multi-Agent Generative Adversarial Imitation Learning](https://mlsanity.com/paper/2510.06913/) (2025)
- [A Gentle Introduction and Tutorial on Deep Generative Models in Transportation Research](https://mlsanity.com/paper/2410.07066/) (2024)
- [DeepIPC: Deeply Integrated Perception and Control for an Autonomous Vehicle in Real Environments](https://mlsanity.com/paper/2207.09934/) (2022)
- [Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Review](https://mlsanity.com/paper/2409.14196/) (2024)
- [Efficient Driving Behavior Narration and Reasoning on Edge Device Using Large Language Models](https://mlsanity.com/paper/2409.20364/) (2024)
- [HAD-Gen: Human-like and Diverse Driving Behavior Modeling for Controllable Scenario Generation](https://mlsanity.com/paper/2503.15049/) (2025)
- [How To Not Drive: Learning Driving Constraints from Demonstration](https://mlsanity.com/paper/2110.00645/) (2021)
- [Fail-Safe Adversarial Generative Imitation Learning](https://mlsanity.com/paper/2203.01696/) (2022)
- [An Adaptive Human Driver Model for Realistic Race Car Simulations](https://mlsanity.com/paper/2203.01909/) (2022)
- [Learning An Active Inference Model of Driver Perception and Control: Application to Vehicle Car-Following](https://mlsanity.com/paper/2303.15201/) (2023)
- [Learning for Long-Horizon Planning via Neuro-Symbolic Abductive Imitation](https://mlsanity.com/paper/2411.18201/) (2024)
- [Modeling Human Behavior Part I -- Learning and Belief Approaches](https://mlsanity.com/paper/2205.06485/) (2022)

## Abstract (arXiv)

An open problem in autonomous vehicle safety validation is building reliable models of human driving behavior in simulation. This work presents an approach to learn neural driving policies from real world driving demonstration data. We model human driving as a sequential decision making problem that is characterized by non-linearity and stochasticity, and unknown underlying cost functions. Imitation learning is an approach for generating intelligent behavior when the cost function is unknown or difficult to specify. Building upon work in inverse reinforcement learning (IRL), Generative Adversarial Imitation Learning (GAIL) aims to provide effective imitation even for problems with large or continuous state and action spaces, such as modeling human driving. This article describes the use of GAIL for learning-based driver modeling. Because driver modeling is inherently a multi-agent problem, where the interaction between agents needs to be modeled, this paper describes a parameter-sharing extension of GAIL called PS-GAIL to tackle multi-agent driver modeling. In addition, GAIL is domain agnostic, making it difficult to encode specific knowledge relevant to driving in the learning process. This paper describes Reward Augmented Imitation Learning (RAIL), which modifies the reward signal to provide domain-specific knowledge to the agent. Finally, human demonstrations are dependent upon latent factors that may not be captured by GAIL. This paper describes Burn-InfoGAIL, which allows for disentanglement of latent variability in demonstrations. Imitation learning experiments are performed using NGSIM, a real-world highway driving dataset. Experiments show that these modifications to GAIL can successfully model highway driving behavior, accurately replicating human demonstrations and generating realistic, emergent behavior in the traffic flow arising from the interaction between driving agents.