From Failures to Fixes: LLM-Driven Scenario Repair for Self-Evolving Autonomous Driving
On the map: Synthetic data and self-improvement for LLMs → Self-evolving LLM agents
Abstract (arXiv)
Ensuring robust and generalizable autonomous driving requires not only broad scenario coverage but also efficient repair of failure cases, particularly those related to challenging and safety-critical scenarios. However, existing scenario generation and selection methods often lack adaptivity and semantic relevance, limiting their impact on performance improvement. In this paper, we propose SERA, an LLM-powered framework that enables autonomous driving systems to self-evolve by repairing failure cases through targeted scenario recommendation. By analyzing performance logs, SERA identifies failure patterns and dynamically retrieves semantically aligned scenarios from a structured bank. An LLM-based reflection mechanism further refines these recommendations to maximize relevance and diversity. The selected scenarios are used for few-shot fine-tuning, enabling targeted adaptation with minimal data. Experiments on the benchmark show that SERA consistently improves key metrics across multiple autonomous driving baselines, demonstrating its effectiveness and generalizability under safety-critical conditions.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Self-evolving LLM agents, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 27 of its 47 references that are in the map. First posted on arXiv 28 May 2025.
Builds on
27 of 47 references.
References · 27
- 2020End-to-End Object Detection with Transformerscited by 7,384
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
- 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
- 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
- 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
- 2021Deep Long-Tailed Learning: A Surveycited by 416
- 2021DSEC: A Stereo Event Camera Dataset for Driving Scenarioscited by 297
- 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
- 2020Frustratingly Simple Few-Shot Object Detectioncited by 280
- 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 247
- 2017One-Shot Imitation Learningcited by 232
- 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
15 more
- 2023Leapfrog Diffusion Model for Stochastic Trajectory Predictioncited by 129
- 2023Language-Guided Traffic Simulation via Scene-Level Diffusioncited by 110
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
- 2022A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspectivecited by 100
- 2021Explanations in Autonomous Driving: A Surveycited by 80
- 2023ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modelingcited by 60
- 2022Meta-DETR: Image-Level Few-Shot Detection with Inter-Class Correlation Exploitationcited by 48
- 2020Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarioscited by 42
- 2020Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologiescited by 41
- 2020Formal Scenario-Based Testing of Autonomous Vehicles: From Simulation to the Real Worldcited by 38
- 2021Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehiclescited by 21
- 2024Enhancing Autonomous Vehicle Training with Language Model Integration and Critical Scenario Generationcited by 19
- 2025Text2Scenario: Text-Driven Scenario Generation for Autonomous Driving Testcited by 18
- 2023SAFR-AV: Safety Analysis of Autonomous Vehicles using Real World Data -- An end-to-end solution for real world data driven scenario-based testing for pre-certification of AV stackscited by 3
- 2024Few-Shot Testing of Autonomous Vehicles with Scenario Similarity Learningcited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Seeking to Collide: Online Safety-Critical Scenario Generation for Autonomous Driving with Retrieval Augmented Large Language Models
- 2026EvoDrive: Pareto Evolution for Safety-Critical Autonomous Driving via Self-Improving LLM Agents
- 2026AnchorDrive: LLM Scenario Rollout with Anchor-Guided Diffusion Regeneration for Safety-Critical Scenario Generation
- 2025SAFE: Harnessing LLM for Scenario-Driven ADS Testing from Multimodal Crash Data ↗
- 2025Learning from Risk: LLM-Guided Generation of Safety-Critical Scenarios with Prior Knowledge
- 2025From Words to Collisions: LLM-Guided Evaluation and Adversarial Generation of Safety-Critical Driving Scenarios
- 2024DRIVE: Dependable Robust Interpretable Visionary Ensemble Framework in Autonomous Driving
- 2025A Tool for Benchmarking Large Language Models' Robustness in Assessing the Realism of Driving Scenarios ↗