# Enhancing Autonomous Vehicle Training with Language Model Integration and Critical Scenario Generation

arXiv: https://arxiv.org/abs/2404.08570 · HTML: https://mlsanity.com/paper/2404.08570/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial scenarios and safety testing, one of 9 directions in End-to-end autonomous driving.
- Builds on 30 of its 45 references that are in the map.
- First posted on arXiv 12 Apr 2024.

## Placed in

- End-to-end autonomous driving / [Adversarial scenarios and safety testing](https://mlsanity.com/driving/adversarial-scenarios-and-safety-testing/)

## Builds on (most cited first, 30 of 30)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [DriveLM: Driving with Graph Visual Question Answering](https://mlsanity.com/paper/2312.14150/) (2023)
- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems](https://mlsanity.com/paper/1810.05642/) (2018)
- [Reward Design with Language Models](https://mlsanity.com/paper/2303.00001/) (2023)
- [DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2309.16292/) (2023)
- [Language Models of Code are Few-Shot Commonsense Learners](https://mlsanity.com/paper/2210.07128/) (2022)
- [Drive Like a Human: Rethinking Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2307.07162/) (2023)
- [A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective](https://mlsanity.com/paper/2202.02215/) (2022)
- [Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs](https://mlsanity.com/paper/1805.05499/) (2018)
- [Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation](https://mlsanity.com/paper/1811.00145/) (2018)
- [Accelerated Evaluation of Automated Vehicles Safety in Lane Change Scenarios Based on Importance Sampling Techniques](https://mlsanity.com/paper/1605.04965/) (2016)
- [Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarios](https://mlsanity.com/paper/2004.06531/) (2020)
- [Augmented LiDAR Simulator for Autonomous Driving](https://mlsanity.com/paper/1811.07112/) (2018)
- [Towards Corner Case Detection for Autonomous Driving](https://mlsanity.com/paper/1902.09184/) (2019)
- [Towards Universal Physical Attacks On Cascaded Camera-Lidar 3D Object Detection Models](https://mlsanity.com/paper/2101.10747/) (2021)
- [Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Review](https://mlsanity.com/paper/2110.08664/) (2021)
- [Extracting Traffic Primitives Directly from Naturalistically Logged Data for Self-Driving Applications](https://mlsanity.com/paper/1709.03553/) (2017)
- [A Comprehensive Review on Ontologies for Scenario-based Testing in the Context of Autonomous Driving](https://mlsanity.com/paper/2304.10837/) (2023)
- [CC-SGG: Corner Case Scenario Generation using Learned Scene Graphs](https://mlsanity.com/paper/2309.09844/) (2023)
- [Highway traffic data: macroscopic, microscopic and criticality analysis for capturing relevant traffic scenarios and traffic modeling based on the highD data set](https://mlsanity.com/paper/1903.04249/) (2019)
- [Identification of Challenging Highway-Scenarios for the Safety Validation of Automated Vehicles Based on Real Driving Data](https://mlsanity.com/paper/2008.11609/) (2020)
- [Bridging Data-Driven and Knowledge-Driven Approaches for Safety-Critical Scenario Generation in Automated Vehicle Validation](https://mlsanity.com/paper/2311.10937/) (2023)
- [Quantitative Risk Indices for Autonomous Vehicle Training Systems](https://mlsanity.com/paper/2104.12945/) (2021)
- [Safe and Efficient Manoeuvring for Emergency Vehicles in Autonomous Traffic using Multi-Agent Proximal Policy Optimisation](https://mlsanity.com/paper/2210.17381/) (2022)

## Built on by (most cited first, 19 of 19)

- [Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysis](https://mlsanity.com/paper/2506.11526/) (2025)
- [Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmap](https://mlsanity.com/paper/2503.21411/) (2025)
- [LLM-attacker: Enhancing Closed-loop Adversarial Scenario Generation for Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2501.15850/) (2025)
- [Realistic Corner Case Generation for Autonomous Vehicles with Multimodal Large Language Model](https://mlsanity.com/paper/2412.00243/) (2024)
- [SEAL: Towards Safe Autonomous Driving via Skill-Enabled Adversary Learning for Closed-Loop Scenario Generation](https://mlsanity.com/paper/2409.10320/) (2024)
- [Preference-based Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2507.14066/) (2025)
- [Enhancing LLM-based Autonomous Driving Agents to Mitigate Perception Attacks](https://mlsanity.com/paper/2409.14488/) (2024)
- [XLM for Autonomous Driving Systems: A Comprehensive Review](https://mlsanity.com/paper/2409.10484/) (2024)
- [AGENTS-LLM: Augmentative GENeration of Challenging Traffic Scenarios with an Agentic LLM Framework](https://mlsanity.com/paper/2507.13729/) (2025)
- [Enhancing Travel Choice Modeling with Large Language Models: A Prompt-Learning Approach](https://mlsanity.com/paper/2406.13558/) (2024)
- [Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Review](https://mlsanity.com/paper/2409.14196/) (2024)
- [A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspective](https://mlsanity.com/paper/2503.23650/) (2025)
- [From Failures to Fixes: LLM-Driven Scenario Repair for Self-Evolving Autonomous Driving](https://mlsanity.com/paper/2505.22067/) (2025)
- [VCAT: Vulnerability-aware and Curiosity-driven Adversarial Training for Enhancing Autonomous Vehicle Robustness](https://mlsanity.com/paper/2409.12997/) (2024)
- [A Novel MLLM-based Approach for Autonomous Driving in Different Weather Conditions](https://mlsanity.com/paper/2411.10603/) (2024)
- [EvoDrive: Pareto Evolution for Safety-Critical Autonomous Driving via Self-Improving LLM Agents](https://mlsanity.com/paper/2606.03678/) (2026)
- [Language-Structured Relational Q-Learning for Threat-Aware Control in Safety-Critical Driving](https://mlsanity.com/paper/2608.11498/) (2026)
- [Ignorance or Incompetence? Constructing Knowledge-Gated, Verifiable Tasks for LLM Agents](https://mlsanity.com/paper/2608.30322/) (2026)
- [REARL: A Closed-loop Autonomous Driving Simulation Enhancement Framework with Real Traffic Data and Large Language Models](https://mlsanity.com/paper/2609.19903/) (2026)

## Abstract (arXiv)

This paper introduces CRITICAL, a novel closed-loop framework for autonomous vehicle (AV) training and testing. CRITICAL stands out for its ability to generate diverse scenarios, focusing on critical driving situations that target specific learning and performance gaps identified in the Reinforcement Learning (RL) agent. The framework achieves this by integrating real-world traffic dynamics, driving behavior analysis, surrogate safety measures, and an optional Large Language Model (LLM) component. It is proven that the establishment of a closed feedback loop between the data generation pipeline and the training process can enhance the learning rate during training, elevate overall system performance, and augment safety resilience. Our evaluations, conducted using the Proximal Policy Optimization (PPO) and the HighwayEnv simulation environment, demonstrate noticeable performance improvements with the integration of critical case generation and LLM analysis, indicating CRITICAL's potential to improve the robustness of AV systems and streamline the generation of critical scenarios. This ultimately serves to hasten the development of AV agents, expand the general scope of RL training, and ameliorate validation efforts for AV safety.