mlsanityData to 14 Oct 2026

Analyzing Closed-loop Training Techniques for Realistic Traffic Agent Models in Autonomous Highway Driving Simulations

arXiv 2410.15987 ↗First posted 21 Oct 2024cs.RO cs.AI cs.LGcited by 1

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

Abstract (arXiv)

Simulation plays a crucial role in the rapid development and safe deployment of autonomous vehicles. Realistic traffic agent models are indispensable for bridging the gap between simulation and the real world. Many existing approaches for imitating human behavior are based on learning from demonstration. However, these approaches are often constrained by focusing on individual training strategies. Therefore, to foster a broader understanding of realistic traffic agent modeling, in this paper, we provide an extensive comparative analysis of different training principles, with a focus on closed-loop methods for highway driving simulation. We experimentally compare (i) open-loop vs. closed-loop multi-agent training, (ii) adversarial vs. deterministic supervised training, (iii) the impact of reinforcement losses, and (iv) the impact of training alongside log-replayed agents to identify suitable training techniques for realistic agent modeling. Furthermore, we identify promising combinations of different closed-loop training methods.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 30 of its 61 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

30 of 61 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios21 Dec
  2. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving18 Oct
  3. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients27 Sep

Other references · 27

  1. 2016Generative Adversarial Imitation Learningcited by 1,597
  2. 2018Which Training Methods for GANs do actually Converge?cited by 679
  3. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 530
  4. 2020Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classificationcited by 506
  5. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  6. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 397
  7. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 252
  8. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 193
  9. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 149
  10. 2020Primal Wasserstein Imitation Learningcited by 121
  11. 2021Learning to drive from a world on railscited by 114
  12. 2023Motion Planning for Autonomous Driving: The State of the Art and Future Perspectivescited by 114
15 more
  1. 2022TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarioscited by 106
  2. 2019Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Datacited by 104
  3. 2018Multi-Agent Generative Adversarial Imitation Learningcited by 100
  4. 2017Imitating Driver Behavior with Generative Adversarial Networkscited by 93
  5. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  6. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observationscited by 77
  7. 2022Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulationcited by 61
  8. 2022DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehiclescited by 37
  9. 2020Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning with Application to Autonomous Drivingcited by 34
  10. 2021Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulationcited by 32
  11. 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
  12. 2019Simulating Emergent Properties of Human Driving Behavior Using Multi-Agent Reward Augmented Imitation Learningcited by 24
  13. 2023Learning Realistic Traffic Agents in Closed-loopcited by 24
  14. 2020Behaviorally Diverse Traffic Simulation via Reinforcement Learningcited by 11
  15. 2021Objective-aware Traffic Simulation via Inverse Reinforcement Learningcited by 8

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025When Planners Meet Reality: How Learned, Reactive Traffic Agents Shift nuPlan Benchmarkscited by 7

Similar, not cited

Close in wording, no citation either way

  1. 2026ForSim: Stepwise Forward Simulation for Traffic Policy Fine-Tuning
  2. 2023Rethinking Closed-loop Training for Autonomous Driving
  3. 2025RoaD: Rollouts as Demonstrations for Closed-Loop Supervised Fine-Tuning of Autonomous Driving Policies
  4. 2024Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Review
  5. 2024Human-compatible driving partners through data-regularized self-play reinforcement learning
  6. 2024CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learning
  7. 2026PCASim: Promptable Closed-loop Adversarial Simulation for Urban Traffic Environment ↗
  8. 2026Imitation Learning for Autonomous Driving in CARLA