Unleashing VLA Potentials in Autonomous Driving via Explicit Learning from Failures
On the map: Vision-language-action models → VLA for autonomous driving
also in 1 more
Abstract (arXiv)
Vision-Language-Action (VLA) models for autonomous driving often hit a performance plateau during Reinforcement Learning (RL) optimization. This stagnation arises from exploration capabilities constrained by previous Supervised Fine-Tuning (SFT), leading to persistent failures in long-tail scenarios. In these critical situations, all explored actions yield a zero-value driving score. This information-sparse reward signals a failure, yet fails to identify its root cause -- whether it is due to incorrect planning, flawed reasoning, or poor trajectory execution. To address this limitation, we propose VLA with Explicit Learning from Failures (ELF-VLA), a framework that augments RL with structured diagnostic feedback. Instead of relying on a vague scalar reward, our method produces detailed, interpretable reports that identify the specific failure mode. The VLA policy then leverages this explicit feedback to generate a Feedback-Guided Refinement. By injecting these corrected, high-reward samples back into the RL training batch, our approach provides a targeted gradient, which enables the policy to solve critical scenarios that unguided exploration cannot. Extensive experiments demonstrate that our method unlocks the latent capabilities of VLA models, achieving state-of-the-art (SOTA) performance on the public NAVSIM benchmark for overall PDMS, EPDMS score and high-level planning accuracy.
Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in VLA for autonomous driving, one of 13 directions in Vision-language-action models. Builds on 40 of its 43 references that are in the map. First posted on arXiv 1 Mar 2026.
Builds on
40 of 43 references.
Closest ancestors · 4
Same direction, newest first
- 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving14 Oct
- 2025A Survey on Vision-Language-Action Models for Autonomous Driving30 Jun
- 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning16 Jun
- 2025Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Models29 May
Other references · 36
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
- 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
- 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
- 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
- 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
- 2025Learning to Reason under Off-Policy Guidancecited by 266
- 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 258
24 more
- 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 244
- 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
- 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
- 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 177
- 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 156
- 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Modelcited by 145
- 2023LingoQA: Visual Question Answering for Autonomous Drivingcited by 144
- 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoningcited by 142
- 2025Pseudo-Simulation for Autonomous Drivingcited by 141
- 2025GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories Generation in End-to-End Autonomous Drivingcited by 117
- 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoningcited by 108
- 2022A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspectivecited by 100
- 2025Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest Questionscited by 95
- 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
- 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 91
- 2025Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedbackcited by 86
- 2024Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Modelscited by 75
- 2022"Curse of rarity" for autonomous vehiclescited by 62
- 2025AdaThinkDrive: Adaptive Thinking via Reinforcement Learning for Autonomous Drivingcited by 57
- 2025Hydra-NeXt: Robust Closed-Loop Driving with Open-Loop Trainingcited by 39
- 2025Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learningcited by 38
- 2025Sce2DriveX: A Generalized MLLM Framework for Scene-to-Drive Learningcited by 25
- 2025ReasonPlan: Unified Scene Prediction and Decision Reasoning for Closed-loop Autonomous Drivingcited by 22
- 2025MTRDrive: Memory-Tool Synergistic Reasoning for Robust Autonomous Driving in Corner Casescited by 9
Built on by
Citing papers per year: 2026: 17.
17 papers.
Sort:
By area:
- 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Drivingcited by 17
- 2026DriveFuture: Future-Aware Latent World Models for Autonomous Drivingcited by 15
- 2026DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scalecited by 12
- 2026EponaV2: Driving World Model with Comprehensive Future Reasoningcited by 12
- 2026Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanationcited by 9
- 2026Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Drivingcited by 2
- 2026Post-Training in End-to-End Autonomous Drivingcited by 2
- 2026DriveReward: A Comprehensive Dataset and Generative Vision-Language Reward Model for Autonomous Drivingcited by 1
- 2026DriveVer: Lightweight Trajectory Evaluator as Test-Time Verifier for Autonomous Drivingcited by 1
- 2026FIRE-VLA: Failure-Informed Self-Evolution for Vision-Language-Action Models in Autonomous Drivingcited by 1
- 2026LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Modelcited by 0
- 2026One Policy, Many Embodiments: Unified Camera-Centric Action Geometry Pre-training for Heterogeneous Embodied Manipulationcited by 0
5 more
- 2026Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Drivingcited by 0
- 2026Evaluation Is All You Need for Multi-Modal Autonomous Drivingcited by 0
- 2026CAR-VLA: Complexity-Aware and Risk-Adaptive Reasoning for Autonomous Drivingcited by 0
- 2026RefineDrive: Reliable Failure-Guided Learning for Vision-Language-Action Drivingcited by 0
- 2026BridgeGuard: Explicit Safety Drift for Diffusion-based Autonomous Drivingcited by 0
- 2026BridgeGuard: Explicit Safety Drift for Diffusion-based Autonomous Driving8 Oct 2026
- 2026RefineDrive: Reliable Failure-Guided Learning for Vision-Language-Action Driving28 Sep 2026
- 2026CAR-VLA: Complexity-Aware and Risk-Adaptive Reasoning for Autonomous Driving28 Sep 2026
- 2026Evaluation Is All You Need for Multi-Modal Autonomous Driving25 Sep 2026
- 2026Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Driving31 Aug 2026
- 2026One Policy, Many Embodiments: Unified Camera-Centric Action Geometry Pre-training for Heterogeneous Embodied Manipulation26 Aug 2026
- 2026FIRE-VLA: Failure-Informed Self-Evolution for Vision-Language-Action Models in Autonomous Driving13 Aug 2026
- 2026Post-Training in End-to-End Autonomous Driving9 Jul 2026
- 2026DriveVer: Lightweight Trajectory Evaluator as Test-Time Verifier for Autonomous Driving1 Jul 2026
- 2026Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Driving22 Jun 2026
- 2026DriveReward: A Comprehensive Dataset and Generative Vision-Language Reward Model for Autonomous Driving7 Jun 2026
- 2026LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model21 May 2026
5 more
- 2026EponaV2: Driving World Model with Comprehensive Future Reasoning14 May 2026
- 2026DriveFuture: Future-Aware Latent World Models for Autonomous Driving10 May 2026
- 2026Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation20 Apr 2026
- 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving2 Apr 2026
- 2026DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scale1 Apr 2026
- 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Drivingcited by 17
- 2026Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Drivingcited by 2
- 2026FIRE-VLA: Failure-Informed Self-Evolution for Vision-Language-Action Models in Autonomous Drivingcited by 1
- 2026One Policy, Many Embodiments: Unified Camera-Centric Action Geometry Pre-training for Heterogeneous Embodied Manipulationcited by 0
- 2026CAR-VLA: Complexity-Aware and Risk-Adaptive Reasoning for Autonomous Drivingcited by 0
- 2026RefineDrive: Reliable Failure-Guided Learning for Vision-Language-Action Drivingcited by 0
- 2026RefineDrive: Reliable Failure-Guided Learning for Vision-Language-Action Driving28 Sep 2026
- 2026CAR-VLA: Complexity-Aware and Risk-Adaptive Reasoning for Autonomous Driving28 Sep 2026
- 2026One Policy, Many Embodiments: Unified Camera-Centric Action Geometry Pre-training for Heterogeneous Embodied Manipulation26 Aug 2026
- 2026FIRE-VLA: Failure-Informed Self-Evolution for Vision-Language-Action Models in Autonomous Driving13 Aug 2026
- 2026Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Driving22 Jun 2026
- 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving2 Apr 2026
- 2026DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scalecited by 12
- 2026Post-Training in End-to-End Autonomous Drivingcited by 2
- 2026DriveReward: A Comprehensive Dataset and Generative Vision-Language Reward Model for Autonomous Drivingcited by 1
- 2026DriveVer: Lightweight Trajectory Evaluator as Test-Time Verifier for Autonomous Drivingcited by 1
- 2026Evaluation Is All You Need for Multi-Modal Autonomous Drivingcited by 0
- 2026Evaluation Is All You Need for Multi-Modal Autonomous Driving25 Sep 2026
- 2026Post-Training in End-to-End Autonomous Driving9 Jul 2026
- 2026DriveVer: Lightweight Trajectory Evaluator as Test-Time Verifier for Autonomous Driving1 Jul 2026
- 2026DriveReward: A Comprehensive Dataset and Generative Vision-Language Reward Model for Autonomous Driving7 Jun 2026
- 2026DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scale1 Apr 2026
- 2026DriveFuture: Future-Aware Latent World Models for Autonomous Drivingcited by 15
- 2026EponaV2: Driving World Model with Comprehensive Future Reasoningcited by 12
- 2026Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanationcited by 9
- 2026LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Modelcited by 0
- 2026LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model21 May 2026
- 2026EponaV2: Driving World Model with Comprehensive Future Reasoning14 May 2026
- 2026DriveFuture: Future-Aware Latent World Models for Autonomous Driving10 May 2026
- 2026Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation20 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2025IRL-VLA: Training an Vision-Language-Action Policy via Reward World Model
- 2025VDRive: Leveraging Reinforced VLA and Diffusion Policy for End-to-end Autonomous Driving
- 2024VLM-RL: A Unified Vision Language Models and Reinforcement Learning Framework for Safe Autonomous Driving
- 2026Learning from Mistakes: Post-Training for Driving VLA with Takeover Data
- 2026Devil is in Narrow Policy: Unleashing Exploration in Driving VLA Models
- 2025AutoDrive-R2: Incentivizing Reasoning and Self-Reflection Capacity for VLA Model in Autonomous Driving
- 2025Reasoning-VLA: A Fast and General Vision-Language-Action Reasoning Model for Autonomous Driving
- 2026DriveVLA-M0: Failure-Aware Memory Augmentation for Autonomous Driving