# CrashAgent: Crash Scenario Generation via Multi-modal Reasoning

arXiv: https://arxiv.org/abs/2505.18341 · HTML: https://mlsanity.com/paper/2505.18341/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning.
- Builds on 23 of its 42 references that are in the map.
- First posted on arXiv 23 May 2025.

## Placed in

- LLM reasoning / [Chain-of-thought reasoning for driving](https://mlsanity.com/lr/chain-of-thought-reasoning-for-driving/)

## Builds on (most cited first, 23 of 23)

- [NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles](https://mlsanity.com/paper/2106.11810/) (2021)
- [Improve Vision Language Model Chain-of-thought Reasoning](https://mlsanity.com/paper/2410.16198/) (2024)
- [Language Conditioned Traffic Generation](https://mlsanity.com/paper/2307.07947/) (2023)
- [Learning Realistic Traffic Agents in Closed-loop](https://mlsanity.com/paper/2311.01394/) (2023)
- [LLM-attacker: Enhancing Closed-loop Adversarial Scenario Generation for Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2501.15850/) (2025)
- [Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehicles](https://mlsanity.com/paper/2409.06450/) (2024)
- [Text2Scenario: Text-Driven Scenario Generation for Autonomous Driving Test](https://mlsanity.com/paper/2503.02911/) (2025)
- [Traffic Scene Generation from Natural Language Description for Autonomous Vehicles with Large Language Model](https://mlsanity.com/paper/2409.09575/) (2024)
- [FREA: Feasibility-Guided Generation of Safety-Critical Scenarios with Reasonable Adversariality](https://mlsanity.com/paper/2406.02983/) (2024)
- [CaDRE: Controllable and Diverse Generation of Safety-Critical Driving Scenarios using Real-World Trajectories](https://mlsanity.com/paper/2403.13208/) (2024)
- [GOOSE: Goal-Conditioned Reinforcement Learning for Safety-Critical Scenario Generation](https://mlsanity.com/paper/2406.03870/) (2024)
- [SoVAR: Building Generalizable Scenarios from Accident Reports for Autonomous Driving Testing](https://mlsanity.com/paper/2409.08081/) (2024)
- [Enhancing Autonomous Driving Safety with Collision Scenario Integration](https://mlsanity.com/paper/2503.03957/) (2025)
- [ReGentS: Real-World Safety-Critical Driving Scenario Generation Made Stable](https://mlsanity.com/paper/2409.07830/) (2024)
- [From Words to Collisions: LLM-Guided Evaluation and Adversarial Generation of Safety-Critical Driving Scenarios](https://mlsanity.com/paper/2502.02145/) (2025)
- [From Dashcam Videos to Driving Simulations: Stress Testing Automated Vehicles against Rare Events](https://mlsanity.com/paper/2411.16027/) (2024)
- [DriveGen: Towards Infinite Diverse Traffic Scenarios with Large Models](https://mlsanity.com/paper/2503.05808/) (2025)
- [Controllable Traffic Simulation through LLM-Guided Hierarchical Reasoning and Refinement](https://mlsanity.com/paper/2409.15135/) (2024)
- [Conversational Code Generation: a Case Study of Designing a Dialogue System for Generating Driving Scenarios for Testing Autonomous Vehicles](https://mlsanity.com/paper/2410.09829/) (2024)
- [Surprise Potential as a Measure of Interactivity in Driving Scenarios](https://mlsanity.com/paper/2502.05677/) (2025)
- [Inherent limitations of LLMs regarding spatial information](https://mlsanity.com/paper/2312.03042/) (2023)
- [Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Review](https://mlsanity.com/paper/2409.14196/) (2024)
- [AccidentSim: Generating Vehicle Collision Videos with Physically Realistic Collision Trajectories from Real-World Accident Reports](https://mlsanity.com/paper/2503.20654/) (2025)

## Built on by (most cited first, 5 of 5)

- [VILTA: A VLM-in-the-Loop Adversary for Enhancing Driving Policy Robustness](https://mlsanity.com/paper/2601.12672/) (2026)
- [The Case for Negative Data: From Crash Reports to Counterfactuals for Reasonable Driving](https://mlsanity.com/paper/2509.18626/) (2025)
- [AITP: Traffic Accident Responsibility Allocation via Multimodal Large Language Models](https://mlsanity.com/paper/2604.20878/) (2026)
- [Learning physically grounded traffic accident reconstruction from public accident reports](https://mlsanity.com/paper/2605.00050/) (2026)
- [Agent-driven Long-tail Simulation for Autonomous Driving](https://mlsanity.com/paper/2607.04331/) (2026)

## Abstract (arXiv)

Training and evaluating autonomous driving algorithms requires a diverse range of scenarios. However, most available datasets predominantly consist of normal driving behaviors demonstrated by human drivers, resulting in a limited number of safety-critical cases. This imbalance, often referred to as a long-tail distribution, restricts the ability of driving algorithms to learn from crucial scenarios involving risk or failure, scenarios that are essential for humans to develop driving skills efficiently. To generate such scenarios, we utilize Multi-modal Large Language Models to convert crash reports of accidents into a structured scenario format, which can be directly executed within simulations. Specifically, we introduce CrashAgent, a multi-agent framework designed to interpret multi-modal real-world traffic crash reports for the generation of both road layouts and the behaviors of the ego vehicle and surrounding traffic participants. We comprehensively evaluate the generated crash scenarios from multiple perspectives, including the accuracy of layout reconstruction, collision rate, and diversity. The resulting high-quality and large-scale crash dataset will be publicly available to support the development of safe driving algorithms in handling safety-critical situations.