mlsanityData to 7 Oct 2026

CarPlanner: Consistent Auto-regressive Trajectory Planning for Large-scale Reinforcement Learning in Autonomous Driving

arXiv 2502.19908 ↗First posted 27 Feb 2025cs.RO cs.CV cs.LGcited by 31

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

Abstract (arXiv)

Trajectory planning is vital for autonomous driving, ensuring safe and efficient navigation in complex environments. While recent learning-based methods, particularly reinforcement learning (RL), have shown promise in specific scenarios, RL planners struggle with training inefficiencies and managing large-scale, real-world driving scenarios. In this paper, we introduce CarPlanner, a Consistent auto-regressive Planner that uses RL to generate multi-modal trajectories. The auto-regressive structure enables efficient large-scale RL training, while the incorporation of consistency ensures stable policy learning by maintaining coherent temporal consistency across time steps. Moreover, CarPlanner employs a generation-selection framework with an expert-guided reward function and an invariant-view module, simplifying RL training and enhancing policy performance. Extensive analysis demonstrates that our proposed RL framework effectively addresses the challenges of training efficiency and performance enhancement, positioning CarPlanner as a promising solution for trajectory planning in autonomous driving. To the best of our knowledge, we are the first to demonstrate that the RL-based planner can surpass both IL- and rule-based state-of-the-arts (SOTAs) on the challenging large-scale real-world dataset nuPlan. Our proposed CarPlanner surpasses RL-, IL-, and rule-based SOTA approaches within this demanding dataset.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 42 of its 53 references that are in the map. First posted on arXiv 27 Feb 2025.

Builds on

42 of 53 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuning8 Oct
  2. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Driving22 Apr
  3. 2023CCIL: Context-conditioned imitation learning for urban driving4 May
  4. 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving10 Mar
  5. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients27 Sep

Other references · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  6. 2022Planning-oriented Autonomous Drivingcited by 1,068
  7. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  8. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  9. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  10. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  11. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  12. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
25 more
  1. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  2. 2019Causal Confusion in Imitation Learningcited by 286
  3. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 267
  4. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 239
  5. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  6. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  7. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
  8. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  9. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  10. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 178
  11. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  12. 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 154
  13. 2020A Survey of End-to-End Driving: Architectures and Training Methodscited by 123
  14. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  15. 2020Model-Augmented Actor-Critic: Backpropagating through Pathscited by 61
  16. 2023ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modelingcited by 60
  17. 2016Model-Free Imitation Learning with Policy Optimizationcited by 52
  18. 2024BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Predictioncited by 49
  19. 2023From Prediction to Planning With Goal Conditioned Lane Graph Traversalscited by 45
  20. 2019Social Attention for Autonomous Decision-Making in Dense Trafficcited by 39
  21. 2023Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priorscited by 36
  22. 2024SmartRefine: A Scenario-Adaptive Refinement Framework for Efficient Motion Predictioncited by 33
  23. 2023Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encodingcited by 32
  24. 2024SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Predictioncited by 25
  25. 2022Accelerating Reinforcement Learning for Autonomous Driving using Task-Agnostic and Ego-Centric Motion Skillscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 21.

31 papers.

Sort:

By area:

  1. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  2. 2025CaRL: Learning Scalable Planning Policies with Simple Rewardscited by 34
  3. 2025IRL-VLA: Training an Vision-Language-Action Policy via Reward World Modelcited by 31
  4. 2025MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learningcited by 25
  5. 2025Plan-R1: Safe and Feasible Trajectory Planning as Language Modelingcited by 19
  6. 2025Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Drivingcited by 19
  7. 2026Generative Scenario Rollouts for End-to-End Autonomous Drivingcited by 6
  8. 2026SimWAM: A Simple World Action Model for End-to-End Autonomous Drivingcited by 5
  9. 2025DAP: A Discrete-token Autoregressive Planner for Autonomous Drivingcited by 4
  10. 2026Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Drivingcited by 4
  11. 2025FlowDrive: moderated flow matching with data balancing for trajectory planningcited by 3
  12. 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspectivecited by 2
19 more
  1. 2025Autoregressive Meta-Actions for Unified Controllable Trajectory Generationcited by 2
  2. 2026FORLER: Federated Offline Reinforcement Learning with Q-Ensemble and Actor Rectificationcited by 2
  3. 2026SCORP: Scene-Consistent Multi-agent Diffusion Planning with Stable Online Reinforcement Post-Training for Cooperative Drivingcited by 2
  4. 2026HEAT: Heterogeneous End-to-End Autonomous Driving via Trajectory-Guided World Modelscited by 2
  5. 2026MTDrive: Multi-turn Interactive Reinforcement Learning for Autonomous Drivingcited by 1
  6. 2026PerlAD: Towards Enhanced Closed-loop End-to-end Autonomous Driving with Pseudo-simulation-based Reinforcement Learningcited by 1
  7. 2026Learning Rollout from Sampling:An R1-Style Tokenized Traffic Simulation Modelcited by 1
  8. 2026SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learningcited by 1
  9. 2026DriveAnchor: Progressive Anchor-based Flow Learning for Autonomous Driving Planningcited by 1
  10. 2026Learning from Mistakes: Rollout-Retrieval Lifelong Policy Learning for Autonomous Drivingcited by 1
  11. 2026World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planningcited by 1
  12. 2026Unified Prediction and Planning via Conflict-Aware Disjoint Parameter Trainingcited by 1
  13. 2026SOAR: Regression-based LiDAR Relocalization for UAVscited by 0
  14. 2026CarPLAN: Context-Adaptive and Robust Planning with Dynamic Scene Awareness for Autonomous Drivingcited by 0
  15. 2026Towards Physically Consistent 4D Scene Reconstruction for Closed-loop Autonomous Driving Simulationcited by 0
  16. 2026FAST: A Framework for Aligned Sampling and Training in Parallel Reinforcement Learning for Autonomous Drivingcited by 0
  17. 2026Not All History Helps: Velocity-Aware Selective Memory for Long-Horizon End-to-End Autonomous Drivingcited by 0
  18. 2026DriveZero: End-to-End Driving Beyond Human Demonstrationscited by 0
  19. 2026GRAVA: Grounded Reasoning-to-Action Representation and Learning for Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026PlanRL: A Trajectory Planning Architecture for Reinforcement Learning-based Driving Experts
  2. 2026RAD-2: Scaling Reinforcement Learning in a Generator-Discriminator Framework
  3. 2026Efficient Multi-Modal Planning with Reward-Guided Preference Optimization for Autonomous Driving
  4. 2024A Reinforcement Learning-Boosted Motion Planning Framework: Comprehensive Generalization Performance in Autonomous Driving
  5. 2025CoPlanner: An Interactive Motion Planner with Contingency-Aware Diffusion for Autonomous Driving
  6. 2026RAPiD: Reward-Guided Consistency Distillation of Diffusion Planners for Real-Time Autonomous Driving
  7. 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidance
  8. 2023Boosting Offline Reinforcement Learning for Autonomous Driving with Hierarchical Latent Skills