# LinguaSim: Interactive Multi-Vehicle Testing Scenario Generation via Natural Language Instruction Based on Large Language Models

arXiv: https://arxiv.org/abs/2510.08046 · HTML: https://mlsanity.com/paper/2510.08046/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 15 references that are in the map.
- First posted on arXiv 9 Oct 2025.

## Builds on (most cited first, 7 of 7)

- [PlanT: Explainable Planning Transformers via Object-Level Representations](https://mlsanity.com/paper/2210.14222/) (2022)
- [ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehicles](https://mlsanity.com/paper/2405.14062/) (2024)
- [ChatSUMO: Large Language Model for Automating Traffic Scenario Generation in Simulation of Urban MObility](https://mlsanity.com/paper/2409.09040/) (2024)
- [Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehicles](https://mlsanity.com/paper/2409.06450/) (2024)
- [Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priors](https://mlsanity.com/paper/2408.03200/) (2024)
- [FREA: Feasibility-Guided Generation of Safety-Critical Scenarios with Reasonable Adversariality](https://mlsanity.com/paper/2406.02983/) (2024)
- [Realistic Corner Case Generation for Autonomous Vehicles with Multimodal Large Language Model](https://mlsanity.com/paper/2412.00243/) (2024)

## Built on by (most cited first, 2 of 2)

- [Chat2Scenic: An Iterative RAG-Based Framework for Scenario Generation in Autonomous Driving](https://mlsanity.com/paper/2607.14387/) (2026)
- [REARL: A Closed-loop Autonomous Driving Simulation Enhancement Framework with Real Traffic Data and Large Language Models](https://mlsanity.com/paper/2609.19903/) (2026)

## Abstract (arXiv)

The generation of testing and training scenarios for autonomous vehicles has drawn significant attention. While Large Language Models (LLMs) have enabled new scenario generation methods, current methods struggle to balance command adherence accuracy with the realism of real-world driving environments. To reduce scenario description complexity, these methods often compromise realism by limiting scenarios to 2D, or open-loop simulations where background vehicles follow predefined, non-interactive behaviors. We propose LinguaSim, an LLM-based framework that converts natural language into realistic, interactive 3D scenarios, ensuring both dynamic vehicle interactions and faithful alignment between the input descriptions and the generated scenarios. A feedback calibration module further refines the generation precision, improving fidelity to user intent. By bridging the gap between natural language and closed-loop, interactive simulations, LinguaSim constrains adversarial vehicle behaviors using both the scenario description and the autonomous driving model guiding them. This framework facilitates the creation of high-fidelity scenarios that enhance safety testing and training. Experiments show LinguaSim can generate scenarios with varying criticality aligned with different natural language descriptions (ACT: 0.072 s for dangerous vs. 3.532 s for safe descriptions; comfortability: 0.654 vs. 0.764), and its refinement module effectively reduces excessive aggressiveness in LinguaSim's initial outputs, lowering the crash rate from 46.9% to 6.3% to better match user intentions.