mlsanityData to 14 Oct 2026

Text2Scenario: Text-Driven Scenario Generation for Autonomous Driving Test

arXiv 2503.02911 ↗First posted 4 Mar 2025cs.SE cs.AI cs.CLcited by 18

On the map: End-to-end autonomous driving → Large language models for driving

Abstract (arXiv)

Autonomous driving (AD) testing constitutes a critical methodology for assessing performance benchmarks prior to product deployment. The creation of segmented scenarios within a simulated environment is acknowledged as a robust and effective strategy; however, the process of tailoring these scenarios often necessitates laborious and time-consuming manual efforts, thereby hindering the development and implementation of AD technologies. In response to this challenge, we introduce Text2Scenario, a framework that leverages a Large Language Model (LLM) to autonomously generate simulation test scenarios that closely align with user specifications, derived from their natural language inputs. Specifically, an LLM, equipped with a meticulously engineered input prompt scheme functions as a text parser for test scenario descriptions, extracting from a hierarchically organized scenario repository the components that most accurately reflect the user's preferences. Subsequently, by exploiting the precedence of scenario components, the process involves sequentially matching and linking scenario representations within a Domain Specific Language corpus, ultimately fabricating executable test scenarios. The experimental results demonstrate that such prompt engineering can meticulously extract the nuanced details of scenario elements embedded within various descriptive formats, with the majority of generated scenarios aligning closely with the user's initial expectations, allowing for the efficient and precise evaluation of diverse AD stacks void of the labor-intensive need for manual scenario configuration. Project page: https://caixxuan.github.io/Text2Scenario.GitHub.io.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Large language models for driving, one of 9 directions in End-to-end autonomous driving. Builds on 27 of its 60 references that are in the map. First posted on arXiv 4 Mar 2025.

Builds on

27 of 60 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Dialogue-based generation of self-driving simulation scenarios using Large Language Models26 Oct

Other references · 26

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  5. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  6. 2022Planning-oriented Autonomous Drivingcited by 1,087
  7. 2024Yi: Open Foundation Models by 01.AIcited by 719
  8. 2022A Systematic Evaluation of Large Language Models of Codecited by 363
  9. 2023Do Large Language Models Know What They Don't Know?cited by 243
  10. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 212
  11. 2020Uncertainty Quantification and Deep Ensemblescited by 139
  12. 2023A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasetscited by 123
14 more
  1. 2023Language-Guided Traffic Simulation via Scene-Level Diffusioncited by 111
  2. 2018Scenic: A Language for Scenario Specification and Scene Generationcited by 83
  3. 2023ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modelingcited by 60
  4. 2018Scenarios for Development, Test and Validation of Automated Vehiclescited by 48
  5. 2020Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarioscited by 43
  6. 2023LLM-BRAIn: AI-driven Fast Generation of Robot Behaviour Tree based on Large Language Modelcited by 24
  7. 2023Robot Behavior-Tree-Based Task Generation with Large Language Modelscited by 23
  8. 2021Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehiclescited by 21
  9. 2021Testing of Autonomous Driving Systems: Where Are We and Where Should We Go?cited by 19
  10. 2022LawBreaker: An Approach for Specifying Traffic Laws and Fuzzing Autonomous Vehiclescited by 16
  11. 2021A Review of Testing Object-Based Environment Perception for Safe Automated Drivingcited by 9
  12. 2021A Scenario-Based Platform for Testing Autonomous Vehicle Behavior Prediction Models in Simulationcited by 6
  13. 2023SAFR-AV: Safety Analysis of Autonomous Vehicles using Real World Data -- An end-to-end solution for real world data driven scenario-based testing for pre-certification of AV stackscited by 3
  14. 2022A Domain-Specific Language for Simulation-Based Testing of IoT Edge-to-Cloud Solutionscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 10.

18 papers.

Sort:

By area:

  1. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 38
  2. 2025Large Language Models and Their Applications in Roadway Safety and Mobility Enhancement: A Comprehensive Reviewcited by 8
  3. 2025Bench2ADVLM: A Closed-Loop Benchmark for Vision-language Models in Autonomous Drivingcited by 8
  4. 2025A Survey of Inductive Reasoning for Large Language Modelscited by 8
  5. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Datacited by 6
  6. 2025CrashAgent: Crash Scenario Generation via Multi-modal Reasoningcited by 5
  7. 2025Adversarial Generation and Collaborative Evolution of Safety-Critical Scenarios for Autonomous Vehiclescited by 5
  8. 2025AirTrafficGen: Configurable Air Traffic Scenario Generation with Large Language Modelscited by 3
  9. 2026Sensor2Sensor: Cross-Embodiment Sensor Conversion for Autonomous Drivingcited by 3
  10. 2025From Failures to Fixes: LLM-Driven Scenario Repair for Self-Evolving Autonomous Drivingcited by 2
  11. 2026SG-CADVLM: A Context-Aware Decoding Powered Vision Language Model for Safety-Critical Scenario Generationcited by 1
  12. 2026Risk-Controllable Multi-View Diffusion for Driving Scenario Generationcited by 1
6 more
  1. 2026Chat2Scenic: An Iterative RAG-Based Framework for Scenario Generation in Autonomous Drivingcited by 1
  2. 2026AdvSerial: Physical Adversarial Attacks on Infrastructure-mounted Pedestrian Detectors via Semantic Feature Suppressioncited by 1
  3. 2026PlannerForge: LLM Agents for Scenario-Based Testing of Motion Planners in Autonomous Drivingcited by 1
  4. 2026GuardAD: Safeguarding Autonomous Driving MLLMs via Markovian Safety Logiccited by 0
  5. 2026From Driving Videos to Simulatable Scenarioscited by 0
  6. 2026Language-Structured Relational Q-Learning for Threat-Aware Control in Safety-Critical Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025David vs. Goliath: A comparative study of different-sized LLMs for code generation in the domain of automotive scenario generation
  2. 2025A Tool for Benchmarking Large Language Models' Robustness in Assessing the Realism of Driving Scenarios ↗
  3. 2025From Words to Collisions: LLM-Guided Evaluation and Adversarial Generation of Safety-Critical Driving Scenarios
  4. 2024Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehicles
  5. 2025SAFE: Harnessing LLM for Scenario-Driven ADS Testing from Multimodal Crash Data ↗
  6. 2024Conversational Code Generation: a Case Study of Designing a Dialogue System for Generating Driving Scenarios for Testing Autonomous Vehicles
  7. 2025Seeking to Collide: Online Safety-Critical Scenario Generation for Autonomous Driving with Retrieval Augmented Large Language Models
  8. 2024Text-to-Drive: Diverse Driving Behavior Synthesis via Large Language Models