mlsanityData to 14 Oct 2026

Seeking to Collide: Online Safety-Critical Scenario Generation for Autonomous Driving with Retrieval Augmented Large Language Models

arXiv 2505.00972 ↗First posted 2 May 2025cs.AI cs.ROcited by 7

On the map: Retrieval-augmented generation → Multimodal RAG

also in 1 more
Abstract (arXiv)

Simulation-based testing is crucial for validating autonomous vehicles (AVs), yet existing scenario generation methods either overfit to common driving patterns or operate in an offline, non-interactive manner that fails to expose rare, safety-critical corner cases. In this paper, we introduce an online, retrieval-augmented large language model (LLM) framework for generating safety-critical driving scenarios. Our method first employs an LLM-based behavior analyzer to infer the most dangerous intent of the background vehicle from the observed state, then queries additional LLM agents to synthesize feasible adversarial trajectories. To mitigate catastrophic forgetting and accelerate adaptation, we augment the framework with a dynamic memorization and retrieval bank of intent-planner pairs, automatically expanding its behavioral library when novel intents arise. Evaluations using the Waymo Open Motion Dataset demonstrate that our model reduces the mean minimum time-to-collision from 1.62 to 1.08 s and incurs a 75% collision rate, substantially outperforming baselines.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 22 of its 28 references that are in the map. First posted on arXiv 2 May 2025.

Builds on

22 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarios19 Dec

Other references · 21

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
  3. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  4. 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Priorcited by 139
  5. 2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 137
  6. 2022TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarioscited by 106
  7. 2022A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspectivecited by 105
  8. 2024ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehiclescited by 93
  9. 2021SceneGen: Learning to Generate Realistic Traffic Scenescited by 84
  10. 2020Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Methodcited by 75
  11. 2022"Curse of rarity" for autonomous vehiclescited by 63
  12. 2023CAT: Closed-loop Adversarial Training for Safe End-to-End Drivingcited by 54
9 more
  1. 2020TrajGAIL: Generating Urban Vehicle Trajectories using Generative Adversarial Imitation Learningcited by 47
  2. 2023Scenario Diffusion: Controllable Driving Scenario Generation With Diffusioncited by 47
  3. 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
  4. 2025Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmapcited by 26
  5. 2025LLM-attacker: Enhancing Closed-loop Adversarial Scenario Generation for Autonomous Driving with Large Language Modelscited by 19
  6. 2024Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehiclescited by 18
  7. 2024Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priorscited by 15
  8. 2024GOOSE: Goal-Conditioned Reinforcement Learning for Safety-Critical Scenario Generationcited by 10
  9. 2024SEAL: Towards Safe Autonomous Driving via Skill-Enabled Adversary Learning for Closed-Loop Scenario Generationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025From Words to Collisions: LLM-Guided Evaluation and Adversarial Generation of Safety-Critical Driving Scenarios
  2. 2026CrashDiffuser: VLM-Guided Collision Intent Reasoning for Fine-Grained Safety-Critical Traffic Scenario Generation
  3. 2026PCASim: Promptable Closed-loop Adversarial Simulation for Urban Traffic Environment ↗
  4. 2025Learning from Risk: LLM-Guided Generation of Safety-Critical Scenarios with Prior Knowledge
  5. 2025LD-Scene: LLM-Guided Diffusion for Controllable Generation of Adversarial Safety-Critical Driving Scenarios
  6. 2025Adversarial Generation and Collaborative Evolution of Safety-Critical Scenarios for Autonomous Vehicles
  7. 2025SAFE: Harnessing LLM for Scenario-Driven ADS Testing from Multimodal Crash Data ↗
  8. 2026SaFeR: Safety-Critical Scenario Generation for Autonomous Driving Test via Feasibility-Constrained Token Resampling