Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Review
On the map: End-to-end autonomous driving → Adversarial scenarios and safety testing
Abstract (arXiv)
Despite advancements in perception and planning for autonomous vehicles (AVs), validating their performance remains a significant challenge. The deployment of planning algorithms in real-world environments is often ineffective due to discrepancies between simulations and real traffic conditions. Evaluating AVs planning algorithms in simulation typically involves replaying driving logs from recorded real-world traffic. However, entities replayed from offline data are not reactive, lack the ability to respond to arbitrary AV behavior, and cannot behave in an adversarial manner to test certain properties of the driving policy. Therefore, simulation with realistic and potentially adversarial entities represents a critical task for AV planning software validation. In this work, we aim to review current research efforts in the field of traffic simulation, focusing on the application of advanced techniques for modeling realistic and adversarial behaviors of traffic entities. The objective of this work is to categorize existing approaches based on the proposed classes of traffic entity behavior and scenario behavior control. Moreover, we collect traffic datasets and examine existing traffic simulations with respect to their employed default traffic entities. Finally, we identify challenges and open questions that hold potential for future research.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Adversarial scenarios and safety testing, one of 9 directions in End-to-end autonomous driving. Builds on 79 of its 119 references that are in the map. First posted on arXiv 21 Sep 2024.
Builds on
79 of 119 references.
Closest ancestors · 6
Same direction, newest first
- 2024SEAL: Towards Safe Autonomous Driving via Skill-Enabled Adversary Learning for Closed-Loop Scenario Generation16 Sep
- 2024Enhancing Autonomous Vehicle Training with Language Model Integration and Critical Scenario Generation12 Apr
- 2024CaDRE: Controllable and Diverse Generation of Safety-Critical Driving Scenarios using Real-World Trajectories19 Mar
- 2023CAT: Closed-loop Adversarial Training for Safe End-to-End Driving19 Oct
- 2023(Re)2H2O: Autonomous Driving Scenario Generation via Reversely Regularized Hybrid Offline-and-Online Reinforcement Learning27 Feb
- 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior9 Dec
Other references · 73
- 2023GPT-4 Technical Reportcited by 18,132
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
- 2016OpenAI Gymcited by 1,684
- 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
- 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
- 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 517
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
- 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
- 2018The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systemscited by 276
- 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 239
38 more
- 2019INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Mapscited by 231
- 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
- 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
- 2022Guided Conditional Diffusion for Controllable Traffic Simulationcited by 189
- 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 158
- 2019The inD Dataset: A Drone Dataset of Naturalistic Road User Trajectories at German Intersectionscited by 137
- 2023Language-Guided Traffic Simulation via Scene-Level Diffusioncited by 110
- 2022TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarioscited by 104
- 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 103
- 2021Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Taskscited by 100
- 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 95
- 2024ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehiclescited by 92
- 2024Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agentscited by 91
- 2023The Waymo Open Sim Agents Challengecited by 86
- 2023Language Conditioned Traffic Generationcited by 86
- 2021SceneGen: Learning to Generate Realistic Traffic Scenescited by 83
- 2021VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehiclescited by 81
- 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
- 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observationscited by 77
- 2019Adaptive Stress Testing for Autonomous Vehiclescited by 70
- 2022Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulationcited by 61
- 2023Trajeglish: Traffic Modeling as Next-Token Predictioncited by 59
- 2022Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real worldcited by 56
- 2023TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Predictioncited by 56
- 2019Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learningcited by 53
- 2018Multi-Agent Imitation Learning for Driving Simulationcited by 49
- 2022AdvDO: Realistic Adversarial Attacks for Trajectory Predictioncited by 48
- 2023RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarioscited by 46
- 2020Modeling Human Driving Behavior through Generative Adversarial Imitation Learningcited by 43
- 2020Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarioscited by 42
- 2019SUMMIT: A Simulator for Urban Driving in Massive Mixed Trafficcited by 39
- 2019Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Drivingcited by 36
- 2024ChatSUMO: Large Language Model for Automating Traffic Scenario Generation in Simulation of Urban MObilitycited by 34
- 2024LimSim++: A Closed-Loop Platform for Deploying Multimodal LLMs in Autonomous Drivingcited by 32
- 2017Sim4CV: A Photo-Realistic Simulator for Computer Vision Applicationscited by 31
- 2020openDD: A Large-Scale Roundabout Drone Datasetcited by 29
- 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
- 2022InterSim: Interactive Traffic Simulation via Explicit Relation Modelingcited by 27
50 most cited of 73 All 73 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Safety-Critical Traffic Simulation with Adversarial Transfer of Driving Intentions
- 2024Data-driven Diffusion Models for Enhancing Safety in Autonomous Vehicle Traffic Simulations
- 2026Real-Time Evaluation of Autonomous Systems under Adversarial Attacks
- 2026Realistic adversarial scenario generation via human-like pedestrian model for autonomous vehicle control parameter optimisation
- 2023SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries
- 2025Generative Modeling for Adversarial Lane-Change Scenarios ↗
- 2024Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priors
- 2024Analyzing Closed-loop Training Techniques for Realistic Traffic Agent Models in Autonomous Highway Driving Simulations