mlsanityData to 14 Oct 2026

Simulating Emergent Properties of Human Driving Behavior Using Multi-Agent Reward Augmented Imitation Learning

arXiv 1903.05766 ↗First posted 14 Mar 2019cs.MA cs.AI cs.LGcited by 24

On the map: Offline RL and imitation learning from data → Imitation and offline RL for driving

Abstract (arXiv)

Recent developments in multi-agent imitation learning have shown promising results for modeling the behavior of human drivers. However, it is challenging to capture emergent traffic behaviors that are observed in real-world datasets. Such behaviors arise due to the many local interactions between agents that are not commonly accounted for in imitation learning. This paper proposes Reward Augmented Imitation Learning (RAIL), which integrates reward augmentation into the multi-agent imitation learning framework and allows the designer to specify prior knowledge in a principled fashion. We prove that convergence guarantees for the imitation learning process are preserved under the application of reward augmentation. This method is validated in a driving scenario, where an entire traffic scene is controlled by driving policies learned using our proposed algorithm. Further, we demonstrate improved performance in comparison to traditional imitation learning algorithms both in terms of the local actions of a single agent and the behavior of emergent properties in complex, multi-agent settings.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data. Builds on 6 of its 22 references that are in the map. First posted on arXiv 14 Mar 2019.

Builds on

6 of 22 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018Multi-Agent Imitation Learning for Driving Simulation2 Mar

Other references · 5

  1. 2015Trust Region Policy Optimizationcited by 2,580
  2. 2016Generative Adversarial Imitation Learningcited by 1,597
  3. 2018Multi-Agent Generative Adversarial Imitation Learningcited by 100
  4. 2017Imitating Driver Behavior with Generative Adversarial Networkscited by 93
  5. 2017RAIL: Risk-Averse Imitation Learningcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 5; 2022: 6; 2023: 4; 2024: 4; 2025: 3; 2026: 1.

24 papers.

Sort:

By area:

  1. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 193
  2. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learningcited by 61
  3. 2020Modeling Human Driving Behavior through Generative Adversarial Imitation Learningcited by 45
  4. 2022TrajGen: Generating Realistic and Diverse Trajectories with Reactive and Feasible Agent Behaviors for Autonomous Drivingcited by 23
  5. 2021Congestion-aware Multi-agent Trajectory Prediction for Collision Avoidancecited by 21
  6. 2023Residual Q-Learning: Offline and Online Policy Customization without Valuecited by 10
  7. 2024LASIL: Learner-Aware Supervised Imitation Learning For Long-term Microscopic Traffic Simulationcited by 8
  8. 2025Diverse Controllable Diffusion Policy with Signal Temporal Logiccited by 8
  9. 2021Reactive and Safe Road User Simulations using Neural Barrier Certificatescited by 7
  10. 2022RITA: Boost Driving Simulators with Realistic Interactive Traffic Flowcited by 7
  11. 2021Grounded Relational Inference: Domain Knowledge Driven Explainable Autonomous Drivingcited by 6
  12. 2022SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environmentscited by 5
12 more
  1. 2025DecompGAIL: Learning Realistic Traffic Behaviors with Decomposed Multi-Agent Generative Adversarial Imitation Learningcited by 5
  2. 2022Exploring the trade off between human driving imitation and safety for traffic simulationcited by 3
  3. 2023ConBaT: Control Barrier Transformer for Safe Policy Learningcited by 3
  4. 2023Long-term Microscopic Traffic Simulation with History-Masked Multi-agent Imitation Learningcited by 3
  5. 2022Fail-Safe Adversarial Generative Imitation Learningcited by 2
  6. 2022Semi-Supervised Imitation Learning of Team Policies from Suboptimal Demonstrationscited by 2
  7. 2024Online Context Learning for Socially Compliant Navigationcited by 2
  8. 2025Hierarchical Imitation Learning of Team Behavior from Heterogeneous Demonstrationscited by 2
  9. 2023INTAGS: Interactive Agent-Guided Simulationcited by 1
  10. 2024LitSim: A Conflict-aware Policy for Long-term Interactive Traffic Simulationcited by 1
  11. 2024Analyzing Closed-loop Training Techniques for Realistic Traffic Agent Models in Autonomous Highway Driving Simulationscited by 1
  12. 2026Artificial Intelligence for Modeling and Simulation of Mixed Automated and Human Trafficcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios
  2. 2025Learning to Drive by Imitating Surrounding Vehicles
  3. 2025Toward Efficient and Robust Behavior Models for Multi-Agent Driving Simulation
  4. 2021Iterative Imitation Policy Improvement for Interactive Autonomous Driving
  5. 2022Learning from Demonstrations of Critical Driving Behaviours Using Driver's Risk Field
  6. 2021Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving
  7. 2023Learning Realistic Traffic Agents in Closed-loop
  8. 2018Deep Imitative Models for Flexible Inference, Planning, and Control