# Validity Learning on Failures: Mitigating the Distribution Shift in Autonomous Vehicle Planning arXiv: https://arxiv.org/abs/2406.01544 ยท HTML: https://mlsanity.com/paper/2406.01544/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. - Builds on 27 of its 45 references that are in the map. - First posted on arXiv 3 Jun 2024. ## Placed in - End-to-end autonomous driving / [Imitation-learning end-to-end planners](https://mlsanity.com/driving/imitation-learning-end-to-end-planners/) - Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/) ## Builds on (most cited first, 27 of 27) - [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017) - [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016) - [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022) - [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022) - [VAD: Vectorized Scene Representation for Efficient Autonomous Driving](https://mlsanity.com/paper/2303.12077/) (2023) - [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020) - [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023) - [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018) - [TNT: Target-driveN Trajectory Prediction](https://mlsanity.com/paper/2008.08294/) (2020) - [A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles](https://mlsanity.com/paper/1604.07446/) (2016) - [Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline](https://mlsanity.com/paper/2206.08129/) (2022) - [MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction](https://mlsanity.com/paper/2111.14973/) (2021) - [Parting with Misconceptions about Learning-based Vehicle Motion Planning](https://mlsanity.com/paper/2306.07962/) (2023) - [DART: Noise Injection for Robust Imitation Learning](https://mlsanity.com/paper/1703.09327/) (2017) - [Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenes](https://mlsanity.com/paper/2305.10430/) (2023) - [Differentiable MPC for End-to-end Planning and Control](https://mlsanity.com/paper/1810.13400/) (2018) - [Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving](https://mlsanity.com/paper/2305.06242/) (2023) - [DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Driving](https://mlsanity.com/paper/2308.00398/) (2023) - [Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios](https://mlsanity.com/paper/2212.11419/) (2022) - [Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients](https://mlsanity.com/paper/2109.13333/) (2021) - [Baidu Apollo EM Motion Planner](https://mlsanity.com/paper/1807.08048/) (2018) - [Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving](https://mlsanity.com/paper/2210.09539/) (2022) - [Learning Realistic Traffic Agents in Closed-loop](https://mlsanity.com/paper/2311.01394/) (2023) - [Limitations and Improvements of the Intelligent Driver Model (IDM)](https://mlsanity.com/paper/2104.02583/) (2021) - [Blending Imitation and Reinforcement Learning for Robust Policy Improvement](https://mlsanity.com/paper/2310.01737/) (2023) - [Analysis of a Modular Autonomous Driving Architecture: The Top Submission to CARLA Leaderboard 2.0 Challenge](https://mlsanity.com/paper/2405.01394/) (2024) - [A Sufficient Condition for Convex Hull Property in General Convex Spatio-Temporal Corridors](https://mlsanity.com/paper/2110.00065/) (2021) ## Built on by (most cited first, 4 of 4) - [ETA: Efficiency through Thinking Ahead, A Dual Approach to Self-Driving with Large Models](https://mlsanity.com/paper/2506.07725/) (2025) - [Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challenges](https://mlsanity.com/paper/2603.09086/) (2026) - [When Environments Shift: Safe Planning with Generative Priors and Robust Conformal Prediction](https://mlsanity.com/paper/2602.12616/) (2026) - [CAPS: Context-Aware Priority Sampling for Enhanced Imitation Learning in Autonomous Driving](https://mlsanity.com/paper/2503.01650/) (2025) ## Abstract (arXiv) The planning problem constitutes a fundamental aspect of the autonomous driving framework. Recent strides in representation learning have empowered vehicles to comprehend their surrounding environments, thereby facilitating the integration of learning-based planning strategies. Among these approaches, Imitation Learning stands out due to its notable training efficiency. However, traditional Imitation Learning methodologies encounter challenges associated with the co-variate shift phenomenon. We propose Validity Learning on Failures, VL(on failure), as a remedy to address this issue. The essence of our method lies in deploying a pre-trained planner across diverse scenarios. Instances where the planner deviates from its immediate objectives, such as maintaining a safe distance from obstacles or adhering to traffic rules, are flagged as failures. The states corresponding to these failures are compiled into a new dataset, termed the failure dataset. Notably, the absence of expert annotations for this data precludes the applicability of standard imitation learning approaches. To facilitate learning from the closed-loop mistakes, we introduce the VL objective which aims to discern valid trajectories within the current environmental context. Experimental evaluations conducted on both reactive CARLA simulation and non-reactive log-replay simulations reveal substantial enhancements in closed-loop metrics such as textitScore, Progress, and Success Rate, underscoring the effectiveness of the proposed methodology. Further evaluations against the Bench2Drive benchmark demonstrate that VL(on failure) outperforms the state-of-the-art methods by a large margin.