mlsanityData to 7 Oct 2026

TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors

arXiv 2101.06557 ↗First posted 17 Jan 2021cs.RO cs.AI cs.CVcited by 192

Not on the map. Why?

Cited by 59 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Simulation has the potential to massively scale evaluation of self-driving systems enabling rapid development as well as safe deployment. To close the gap between simulation and the real world, we need to simulate realistic multi-agent behaviors. Existing simulation environments rely on heuristic-based models that directly encode traffic rules, which cannot capture irregular maneuvers (e.g., nudging, U-turns) and complex interactions (e.g., yielding, merging). In contrast, we leverage real-world data to learn directly from human demonstration and thus capture a more diverse set of actor behaviors. To this end, we propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy that generates socially-consistent plans for all actors in the scene jointly. To learn a robust policy amenable for long horizon simulation, we unroll the policy in training and optimize through the fully differentiable simulation across time. Our learning objective incorporates both human demonstrations as well as common sense. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines. Notably, we can exploit trajectories generated by TrafficSim as effective data augmentation for training better motion planner.

Key facts. Cited by 192 papers in the map (data through 7 Oct 2026). Builds on 23 of its 46 references that are in the map. First posted on arXiv 17 Jan 2021.

Builds on

23 of 46 references.

References · 23

  1. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  2. 2016Generative Adversarial Imitation Learningcited by 1,576
  3. 2016End to End Learning for Self-Driving Carscited by 1,000
  4. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  5. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  6. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  7. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  8. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  9. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 178
  10. 2019Multiple Futures Predictioncited by 178
  11. 2017Arbitrary-Oriented Scene Text Detection via Rotation Proposalscited by 176
  12. 2020LiDARsim: Realistic LiDAR Simulation by Leveraging the Real Worldcited by 137
11 more
  1. 2020Implicit Latent Variable Model for Scene-Consistent Motion Forecastingcited by 116
  2. 2019Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Datacited by 104
  3. 2019Diverse Trajectory Forecasting with Determinantal Point Processescited by 80
  4. 2020End-to-end Contextual Perception and Prediction with Interaction Transformercited by 54
  5. 2019Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Trafficcited by 50
  6. 2018Multi-Agent Imitation Learning for Driving Simulationcited by 49
  7. 2020Modeling Human Driving Behavior through Generative Adversarial Imitation Learningcited by 43
  8. 2017Beyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving Carscited by 26
  9. 2018Learning from Demonstration in the Wildcited by 25
  10. 2020The Importance of Prior Knowledge in Precise Multimodal Predictioncited by 25
  11. 2019Simulating Emergent Properties of Human Driving Behavior Using Multi-Agent Reward Augmented Imitation Learningcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 29; 2023: 42; 2024: 46; 2025: 46; 2026: 18.

192 papers.

Sort:

By area:

  1. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  2. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  3. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  4. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  5. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  6. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  7. 2022Guided Conditional Diffusion for Controllable Traffic Simulationcited by 189
  8. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
  9. 2022V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Drivingcited by 157
  10. 2023V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perceptioncited by 155
  11. 2023Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusioncited by 137
  12. 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Priorcited by 136
38 more
  1. 2022Learning from All Vehiclescited by 135
  2. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  3. 2021Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Predictioncited by 131
  4. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
  5. 2023Language-Guided Traffic Simulation via Scene-Level Diffusioncited by 110
  6. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 95
  7. 2023The Waymo Open Sim Agents Challengecited by 86
  8. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  9. 2023FusionAD: Multi-modality Fusion for Prediction and Planning Tasks of Autonomous Drivingcited by 67
  10. 2025Robust Autonomy Emerges from Self-Playcited by 64
  11. 2022Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulationcited by 61
  12. 2025A Survey of World Models for Autonomous Drivingcited by 60
  13. 2023Trajeglish: Traffic Modeling as Next-Token Predictioncited by 59
  14. 2022Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real worldcited by 56
  15. 2023TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Predictioncited by 56
  16. 2024BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Predictioncited by 49
  17. 2022AdvDO: Realistic Adversarial Attacks for Trajectory Predictioncited by 48
  18. 2024SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rolloutcited by 47
  19. 2023Scenario Diffusion: Controllable Driving Scenario Generation With Diffusioncited by 46
  20. 2023RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarioscited by 46
  21. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Drivingcited by 43
  22. 2024Solving Motion Planning Tasks with a Scalable Generative Modelcited by 40
  23. 2024Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Modelscited by 40
  24. 2022DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehiclescited by 36
  25. 2023SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversariescited by 36
  26. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 36
  27. 2021Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulationcited by 32
  28. 2023Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encodingcited by 32
  29. 2024WcDT: World-centric Diffusion Transformer for Traffic Scene Generationcited by 31
  30. 2025CarPlanner: Consistent Auto-regressive Trajectory Planning for Large-scale Reinforcement Learning in Autonomous Drivingcited by 31
  31. 2025MoFlow: One-Step Flow Matching for Human Trajectory Forecasting via Implicit Maximum Likelihood Estimation based Distillationcited by 31
  32. 2021Autonomy 2.0: Why is self-driving always 5 years away?cited by 30
  33. 2022Behavioral Intention Prediction in Driving Scenes: A Surveycited by 30
  34. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  35. 2021PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulationcited by 29
  36. 2023Rethinking Closed-loop Training for Autonomous Drivingcited by 29
  37. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  38. 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29

50 most cited of 192 All 192 (JSON)