# CarPlanner: Consistent Auto-regressive Trajectory Planning for Large-scale Reinforcement Learning in Autonomous Driving

arXiv: https://arxiv.org/abs/2502.19908 · HTML: https://mlsanity.com/paper/2502.19908/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving.
- Builds on 42 of its 53 references that are in the map.
- First posted on arXiv 27 Feb 2025.

## Placed in

- End-to-end autonomous driving / [Imitation-learning end-to-end planners](https://mlsanity.com/driving/imitation-learning-end-to-end-planners/)
- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020)
- [NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles](https://mlsanity.com/paper/2106.11810/) (2021)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving](https://mlsanity.com/paper/2205.15997/) (2022)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [Causal Confusion in Imitation Learning](https://mlsanity.com/paper/1905.11979/) (2019)
- [Motion Transformer with Global Intention Localization and Local Movement Refinement](https://mlsanity.com/paper/2209.13508/) (2022)
- [Parting with Misconceptions about Learning-based Vehicle Motion Planning](https://mlsanity.com/paper/2306.07962/) (2023)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusion](https://mlsanity.com/paper/2306.03083/) (2023)
- [MP3: A Unified Model to Map, Perceive, Predict and Plan](https://mlsanity.com/paper/2101.06806/) (2021)
- [TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors](https://mlsanity.com/paper/2101.06557/) (2021)
- [PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settings](https://mlsanity.com/paper/1905.01296/) (2019)
- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [MotionLM: Multi-Agent Motion Forecasting as Language Modeling](https://mlsanity.com/paper/2309.16534/) (2023)
- [GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving](https://mlsanity.com/paper/2303.05760/) (2023)
- [A Survey of End-to-End Driving: Architectures and Training Methods](https://mlsanity.com/paper/2003.06404/) (2020)
- [Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients](https://mlsanity.com/paper/2109.13333/) (2021)
- [PlanT: Explainable Planning Transformers via Object-Level Representations](https://mlsanity.com/paper/2210.14222/) (2022)
- [PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Driving](https://mlsanity.com/paper/2404.14327/) (2024)
- [Model-Augmented Actor-Critic: Backpropagating through Paths](https://mlsanity.com/paper/2005.08068/) (2020)

## Built on by (most cited first, 30 of 34)

- [ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving](https://mlsanity.com/paper/2506.08052/) (2025)
- [CaRL: Learning Scalable Planning Policies with Simple Rewards](https://mlsanity.com/paper/2504.17838/) (2025)
- [IRL-VLA: Training an Vision-Language-Action Policy via Reward World Model](https://mlsanity.com/paper/2508.06571/) (2025)
- [MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learning](https://mlsanity.com/paper/2512.13636/) (2025)
- [Plan-R1: Safe and Feasible Trajectory Planning as Language Modeling](https://mlsanity.com/paper/2505.17659/) (2025)
- [Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Driving](https://mlsanity.com/paper/2506.09800/) (2025)
- [SimWAM: A Simple World Action Model for End-to-End Autonomous Driving](https://mlsanity.com/paper/2608.07468/) (2026)
- [Generative Scenario Rollouts for End-to-End Autonomous Driving](https://mlsanity.com/paper/2601.11475/) (2026)
- [FlowDrive: moderated flow matching with data balancing for trajectory planning](https://mlsanity.com/paper/2509.21961/) (2025)
- [DAP: A Discrete-token Autoregressive Planner for Autonomous Driving](https://mlsanity.com/paper/2511.13306/) (2025)
- [Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving](https://mlsanity.com/paper/2603.13842/) (2026)
- [DriveZero: End-to-End Driving Beyond Human Demonstrations](https://mlsanity.com/paper/2609.06055/) (2026)
- [A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspective](https://mlsanity.com/paper/2503.23650/) (2025)
- [Autoregressive Meta-Actions for Unified Controllable Trajectory Generation](https://mlsanity.com/paper/2505.23612/) (2025)
- [BAP-SRL: Bayesian Adaptive Priority Safe Reinforcement Learning for Vehicle Motion Planning at Mixed Traffic Intersections](https://mlsanity.com/paper/2601.21679/) (2026)
- [MTDrive: Multi-turn Interactive Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2601.22930/) (2026)
- [FORLER: Federated Offline Reinforcement Learning with Q-Ensemble and Actor Rectification](https://mlsanity.com/paper/2602.02055/) (2026)
- [SCORP: Scene-Consistent Multi-agent Diffusion Planning with Stable Online Reinforcement Post-Training for Cooperative Driving](https://mlsanity.com/paper/2604.11734/) (2026)
- [HEAT: Heterogeneous End-to-End Autonomous Driving via Trajectory-Guided World Models](https://mlsanity.com/paper/2605.19631/) (2026)
- [PerlAD: Towards Enhanced Closed-loop End-to-end Autonomous Driving with Pseudo-simulation-based Reinforcement Learning](https://mlsanity.com/paper/2603.14908/) (2026)
- [Learning Rollout from Sampling:An R1-Style Tokenized Traffic Simulation Model](https://mlsanity.com/paper/2603.24989/) (2026)
- [SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learning](https://mlsanity.com/paper/2604.24729/) (2026)
- [DriveAnchor: Progressive Anchor-based Flow Learning for Autonomous Driving Planning](https://mlsanity.com/paper/2606.00519/) (2026)
- [Learning from Mistakes: Rollout-Retrieval Lifelong Policy Learning for Autonomous Driving](https://mlsanity.com/paper/2606.30537/) (2026)
- [World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning](https://mlsanity.com/paper/2607.10630/) (2026)
- [Unified Prediction and Planning via Conflict-Aware Disjoint Parameter Training](https://mlsanity.com/paper/2607.19971/) (2026)
- [SimVTA: Simulation-Grounded Video-Trajectory Alignment for Driving World Action Models](https://mlsanity.com/paper/2610.13781/) (2026)
- [SOAR: Regression-based LiDAR Relocalization for UAVs](https://mlsanity.com/paper/2602.13267/) (2026)
- [CarPLAN: Context-Adaptive and Robust Planning with Dynamic Scene Awareness for Autonomous Driving](https://mlsanity.com/paper/2603.12607/) (2026)
- [Towards Physically Consistent 4D Scene Reconstruction for Closed-loop Autonomous Driving Simulation](https://mlsanity.com/paper/2605.21032/) (2026)

## Abstract (arXiv)

Trajectory planning is vital for autonomous driving, ensuring safe and efficient navigation in complex environments. While recent learning-based methods, particularly reinforcement learning (RL), have shown promise in specific scenarios, RL planners struggle with training inefficiencies and managing large-scale, real-world driving scenarios. In this paper, we introduce CarPlanner, a Consistent auto-regressive Planner that uses RL to generate multi-modal trajectories. The auto-regressive structure enables efficient large-scale RL training, while the incorporation of consistency ensures stable policy learning by maintaining coherent temporal consistency across time steps. Moreover, CarPlanner employs a generation-selection framework with an expert-guided reward function and an invariant-view module, simplifying RL training and enhancing policy performance. Extensive analysis demonstrates that our proposed RL framework effectively addresses the challenges of training efficiency and performance enhancement, positioning CarPlanner as a promising solution for trajectory planning in autonomous driving. To the best of our knowledge, we are the first to demonstrate that the RL-based planner can surpass both IL- and rule-based state-of-the-arts (SOTAs) on the challenging large-scale real-world dataset nuPlan. Our proposed CarPlanner surpasses RL-, IL-, and rule-based SOTA approaches within this demanding dataset.