mlsanityData to 7 Oct 2026

CAPS: Context-Aware Priority Sampling for Enhanced Imitation Learning in Autonomous Driving

arXiv 2503.01650 ↗First posted 3 Mar 2025cs.LG cs.ROcited by 0

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

also in 1 more
Abstract (arXiv)

In this paper, we introduce Context-Aware Priority Sampling (CAPS), a novel method designed to enhance data efficiency in learning-based autonomous driving systems. CAPS addresses the challenge of imbalanced datasets in imitation learning by leveraging Vector Quantized Variational Autoencoders (VQ-VAEs). In this way, we can get structured and interpretable data representations, which help to reveal meaningful patterns in the data. These patterns are used to group the data into clusters, with each sample being assigned a cluster ID. The cluster IDs are then used to re-balance the dataset, ensuring that rare yet valuable samples receive higher priority during training. We evaluate our method through closed-loop experiments in the CARLA simulator. The results on Bench2Drive scenarios demonstrate the effectiveness of CAPS in enhancing model generalization, with substantial improvements in both driving score and success rate.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 37 of its 49 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

37 of 49 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024CIMRL: Combining IMitation and Reinforcement Learning for Safe Autonomous Driving13 Jun
  2. 2024Validity Learning on Failures: Mitigating the Distribution Shift in Autonomous Vehicle Planning3 Jun
  3. 2024Augmenting Safety-Critical Driving Scenarios while Preserving Similarity to Expert Trajectories20 Apr
  4. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios21 Dec
  5. 2022Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot Curricula2 Dec
  6. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving18 Oct

Other references · 31

  1. 2017Neural Discrete Representation Learningcited by 4,650
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  5. 2015Prioritized Experience Replaycited by 1,070
  6. 2022Planning-oriented Autonomous Drivingcited by 1,068
  7. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  8. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  9. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
  10. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  11. 2020TNT: Target-driveN Trajectory Predictioncited by 345
  12. 2019Causal Confusion in Imitation Learningcited by 286
19 more
  1. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 250
  2. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 247
  3. 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
  4. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  5. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 142
  6. 2018EnsembleDAgger: A Bayesian Approach to Safe Imitation Learningcited by 96
  7. 2023Input Perturbation Reduces Exposure Bias in Diffusion Modelscited by 88
  8. 2016Query-Efficient Imitation Learning for End-to-End Autonomous Drivingcited by 83
  9. 2023Model-Based Runtime Monitoring with Interactive Imitation Learningcited by 37
  10. 2023Learning Realistic Traffic Agents in Closed-loopcited by 24
  11. 2020Combined Cleaning and Resampling Algorithm for Multi-Class Imbalanced Data with Label Noisecited by 18
  12. 2024The Why, When, and How to Use Active Learning in Large-Data-Driven 3D Object Detection for Safe Autonomous Driving: An Empirical Explorationcited by 17
  13. 2023Blending Imitation and Reinforcement Learning for Robust Policy Improvementcited by 16
  14. 2024ActiveAD: Planning-Oriented Active Learning for End-to-End Autonomous Drivingcited by 16
  15. 2024Curb Your Attention: Causal Attention Gating for Robust Trajectory Prediction in Autonomous Drivingcited by 9
  16. 2021Solving the Class Imbalance Problem Using a Counterfactual Method for Data Augmentationcited by 8
  17. 2024Analysis of a Modular Autonomous Driving Architecture: The Top Submission to CARLA Leaderboard 2.0 Challengecited by 8
  18. 2020Sample Efficient Interactive End-to-End Deep Learning for Self-Driving Cars with Selective Multi-Class Safe Dataset Aggregationcited by 7
  19. 2025Getting SMARTER for Motion Planning in Autonomous Driving Systemscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025V-Max: A Reinforcement Learning Framework for Autonomous Driving
  2. 2025Learning to Drive by Imitating Surrounding Vehicles
  3. 2023Specification-Guided Data Aggregation for Semantically Aware Imitation Learning
  4. 2024LASIL: Learner-Aware Supervised Imitation Learning For Long-term Microscopic Traffic Simulation
  5. 2021Exploring Imitation Learning for Autonomous Driving with Feedback Synthesizer and Differentiable Rasterization
  6. 2019Safer End-to-End Autonomous Driving via Conditional Imitation Learning and Command Augmentation
  7. 2024Good Data Is All Imitation Learning Needs
  8. 2023CCIL: Context-conditioned imitation learning for urban driving