mlsanityData to 7 Oct 2026

Conversational Code Generation: a Case Study of Designing a Dialogue System for Generating Driving Scenarios for Testing Autonomous Vehicles

arXiv 2410.09829 ↗First posted 13 Oct 2024cs.CL cs.IR cs.ROcited by 4

On the map: End-to-end autonomous driving → Large language models for driving

Abstract (arXiv)

Cyber-physical systems like autonomous vehicles are tested in simulation before deployment, using domain-specific programs for scenario specification. To aid the testing of autonomous vehicles in simulation, we design a natural language interface, using an instruction-following large language model, to assist a non-coding domain expert in synthesising the desired scenarios and vehicle behaviours. We show that using it to convert utterances to the symbolic program is feasible, despite the very small training dataset. Human experiments show that dialogue is critical to successful simulation generation, leading to a 4.5 times higher success rate than a generation without engaging in extended conversation.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Large language models for driving, one of 9 directions in End-to-end autonomous driving. Builds on 20 of its 37 references that are in the map. First posted on arXiv 13 Oct 2024.

Builds on

20 of 37 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehicles22 May
  2. 2023Dialogue-based generation of self-driving simulation scenarios using Large Language Models26 Oct

Other references · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2023Mistral 7Bcited by 3,065
  5. 2024A Survey on Large Language Models for Code Generationcited by 646
  6. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 468
  7. 2022The Stack: 3 TB of permissively licensed source codecited by 354
  8. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
  9. 2017A Survey on Dialogue Systems: Recent Advances and New Frontierscited by 162
  10. 2016Language to Logical Form with Neural Attentioncited by 158
  11. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  12. 2022TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarioscited by 104
6 more
  1. 2020A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systemscited by 71
  2. 2023ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modelingcited by 60
  3. 2020Scenic: A Language for Scenario Specification and Data Generationcited by 39
  4. 2023FRANC: A Lightweight Framework for High-Quality Code Generationcited by 14
  5. 2023Exploring Data Augmentation for Code Generation Taskscited by 7
  6. 2021Memory-Based Semantic Parsingcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025LLM-Based Multi-Agent Blackboard System for Information Discovery in Data Sciencecited by 15
  2. 2025CrashAgent: Crash Scenario Generation via Multi-modal Reasoningcited by 5
  3. 2025David vs. Goliath: A comparative study of different-sized LLMs for code generation in the domain of automotive scenario generationcited by 1
  4. 2026Chat2Scenic: An Iterative RAG-Based Framework for Scenario Generation in Autonomous Drivingcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Text2Scenario: Text-Driven Scenario Generation for Autonomous Driving Test
  2. 2022DOROTHIE: Spoken Dialogue for Handling Unexpected Situations in Interactive Autonomous Driving Agents
  3. 2024ScenicNL: Generating Probabilistic Scenario Programs from Natural Language
  4. 2025ChatGPT on the Road: Leveraging Large Language Model-Powered In-vehicle Conversational Agents for Safer and More Enjoyable Driving Experience
  5. 2026Modular Autonomy with Conversational Interaction: An LLM-driven Framework for Decision Making in Autonomous Driving
  6. 2025CoopReflect: Towards Natural Language Communication for Cooperative Autonomous Driving via Multi-Agent Learning
  7. 2023Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles
  8. 2025A Tool for Benchmarking Large Language Models' Robustness in Assessing the Realism of Driving Scenarios ↗