# Reality Bites: Assessing the Realism of Driving Scenarios with Large Language Models

arXiv: https://arxiv.org/abs/2403.09906 · HTML: https://mlsanity.com/paper/2403.09906/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Mar 2024.

## Built on by (most cited first, 11 of 11)

- [Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysis](https://mlsanity.com/paper/2506.11526/) (2025)
- [From Words to Collisions: LLM-Guided Evaluation and Adversarial Generation of Safety-Critical Driving Scenarios](https://mlsanity.com/paper/2502.02145/) (2025)
- [Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward](https://mlsanity.com/paper/2404.08517/) (2024)
- [Benchmarking Image Perturbations for Testing Automated Driving Assistance Systems](https://mlsanity.com/paper/2501.12269/) (2025)
- [Feature-Guided Analysis of Neural Networks: A Replication Study](https://mlsanity.com/paper/2511.00052/) (2025)
- [Assessing the Uncertainty and Robustness of the Laptop Refurbishing Software](https://mlsanity.com/paper/2409.03782/) (2024)
- [Generating Critical Scenarios for Testing Automated Driving Systems](https://mlsanity.com/paper/2412.02574/) (2024)
- [A Multi-Modality Evaluation of the Reality Gap in Autonomous Driving Systems](https://mlsanity.com/paper/2509.22379/) (2025)
- [SciQu: Accelerating Materials Properties Prediction with Automated Literature Mining for Self-Driving Laboratories](https://mlsanity.com/paper/2407.08270/) (2024)
- [SAFLITE: Fuzzing Autonomous Systems via Large Language Models](https://mlsanity.com/paper/2412.18727/) (2024)
- [Search-Based Software Engineering and AI Foundation Models: Current Landscape and Future Roadmap](https://mlsanity.com/paper/2505.19625/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) are demonstrating outstanding potential for tasks such as text generation, summarization, and classification. Given that such models are trained on a humongous amount of online knowledge, we hypothesize that LLMs can assess whether driving scenarios generated by autonomous driving testing techniques are realistic, i.e., being aligned with real-world driving conditions. To test this hypothesis, we conducted an empirical evaluation to assess whether LLMs are effective and robust in performing the task. This reality check is an important step towards devising LLM-based autonomous driving testing techniques. For our empirical evaluation, we selected 64 realistic scenarios from deepscenario--an open driving scenario dataset. Next, by introducing minor changes to them, we created 512 additional realistic scenarios, to form an overall dataset of 576 scenarios. With this dataset, we evaluated three LLMs (gpt, llama, and mistral) to assess their robustness in assessing the realism of driving scenarios. Our results show that: (1) Overall, gpt achieved the highest robustness compared to llama and mistral, consistently throughout almost all scenarios, roads, and weather conditions; (2) mistral performed the worst consistently; (3) llama achieved good results under certain conditions; and (4) roads and weather conditions do influence the robustness of the LLMs.