mlsanityData to 7 Oct 2026

Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Review

arXiv 2409.14196 ↗First posted 21 Sep 2024cs.ROcited by 3

On the map: End-to-end autonomous driving → Adversarial scenarios and safety testing

Abstract (arXiv)

Despite advancements in perception and planning for autonomous vehicles (AVs), validating their performance remains a significant challenge. The deployment of planning algorithms in real-world environments is often ineffective due to discrepancies between simulations and real traffic conditions. Evaluating AVs planning algorithms in simulation typically involves replaying driving logs from recorded real-world traffic. However, entities replayed from offline data are not reactive, lack the ability to respond to arbitrary AV behavior, and cannot behave in an adversarial manner to test certain properties of the driving policy. Therefore, simulation with realistic and potentially adversarial entities represents a critical task for AV planning software validation. In this work, we aim to review current research efforts in the field of traffic simulation, focusing on the application of advanced techniques for modeling realistic and adversarial behaviors of traffic entities. The objective of this work is to categorize existing approaches based on the proposed classes of traffic entity behavior and scenario behavior control. Moreover, we collect traffic datasets and examine existing traffic simulations with respect to their employed default traffic entities. Finally, we identify challenges and open questions that hold potential for future research.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Adversarial scenarios and safety testing, one of 9 directions in End-to-end autonomous driving. Builds on 79 of its 119 references that are in the map. First posted on arXiv 21 Sep 2024.

Builds on

79 of 119 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024SEAL: Towards Safe Autonomous Driving via Skill-Enabled Adversary Learning for Closed-Loop Scenario Generation16 Sep
  2. 2024Enhancing Autonomous Vehicle Training with Language Model Integration and Critical Scenario Generation12 Apr
  3. 2024CaDRE: Controllable and Diverse Generation of Safety-Critical Driving Scenarios using Real-World Trajectories19 Mar
  4. 2023CAT: Closed-loop Adversarial Training for Safe End-to-End Driving19 Oct
  5. 2023(Re)2H2O: Autonomous Driving Scenario Generation via Reversely Regularized Hybrid Offline-and-Online Reinforcement Learning27 Feb
  6. 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior9 Dec

Other references · 73

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  5. 2016OpenAI Gymcited by 1,684
  6. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  7. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
  8. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 517
  9. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  10. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
  11. 2018The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systemscited by 276
  12. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 239
38 more
  1. 2019INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Mapscited by 231
  2. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
  3. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  4. 2022Guided Conditional Diffusion for Controllable Traffic Simulationcited by 189
  5. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 158
  6. 2019The inD Dataset: A Drone Dataset of Naturalistic Road User Trajectories at German Intersectionscited by 137
  7. 2023Language-Guided Traffic Simulation via Scene-Level Diffusioncited by 110
  8. 2022TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarioscited by 104
  9. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 103
  10. 2021Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Taskscited by 100
  11. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 95
  12. 2024ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehiclescited by 92
  13. 2024Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agentscited by 91
  14. 2023The Waymo Open Sim Agents Challengecited by 86
  15. 2023Language Conditioned Traffic Generationcited by 86
  16. 2021SceneGen: Learning to Generate Realistic Traffic Scenescited by 83
  17. 2021VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehiclescited by 81
  18. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  19. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observationscited by 77
  20. 2019Adaptive Stress Testing for Autonomous Vehiclescited by 70
  21. 2022Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulationcited by 61
  22. 2023Trajeglish: Traffic Modeling as Next-Token Predictioncited by 59
  23. 2022Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real worldcited by 56
  24. 2023TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Predictioncited by 56
  25. 2019Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learningcited by 53
  26. 2018Multi-Agent Imitation Learning for Driving Simulationcited by 49
  27. 2022AdvDO: Realistic Adversarial Attacks for Trajectory Predictioncited by 48
  28. 2023RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarioscited by 46
  29. 2020Modeling Human Driving Behavior through Generative Adversarial Imitation Learningcited by 43
  30. 2020Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarioscited by 42
  31. 2019SUMMIT: A Simulator for Urban Driving in Massive Mixed Trafficcited by 39
  32. 2019Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Drivingcited by 36
  33. 2024ChatSUMO: Large Language Model for Automating Traffic Scenario Generation in Simulation of Urban MObilitycited by 34
  34. 2024LimSim++: A Closed-Loop Platform for Deploying Multimodal LLMs in Autonomous Drivingcited by 32
  35. 2017Sim4CV: A Photo-Realistic Simulator for Computer Vision Applicationscited by 31
  36. 2020openDD: A Large-Scale Roundabout Drone Datasetcited by 29
  37. 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
  38. 2022InterSim: Interactive Traffic Simulation via Explicit Relation Modelingcited by 27

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Safety-Critical Traffic Simulation with Adversarial Transfer of Driving Intentions
  2. 2024Data-driven Diffusion Models for Enhancing Safety in Autonomous Vehicle Traffic Simulations
  3. 2026Real-Time Evaluation of Autonomous Systems under Adversarial Attacks
  4. 2026Realistic adversarial scenario generation via human-like pedestrian model for autonomous vehicle control parameter optimisation
  5. 2023SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries
  6. 2025Generative Modeling for Adversarial Lane-Change Scenarios ↗
  7. 2024Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priors
  8. 2024Analyzing Closed-loop Training Techniques for Realistic Traffic Agent Models in Autonomous Highway Driving Simulations