# DriveGPT: Scaling Autoregressive Behavior Models for Driving

arXiv: https://arxiv.org/abs/2412.14415 · HTML: https://mlsanity.com/paper/2412.14415/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving.
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 19 Dec 2024.

## Placed in

- End-to-end autonomous driving / [Imitation-learning end-to-end planners](https://mlsanity.com/driving/imitation-learning-end-to-end-planners/)

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020)
- [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://mlsanity.com/paper/2012.00955/) (2020)
- [Scaling Data-Constrained Language Models](https://mlsanity.com/paper/2305.16264/) (2023)
- [Learning Lane Graph Representations for Motion Forecasting](https://mlsanity.com/paper/2007.13732/) (2020)
- [Wayformer: Motion Forecasting via Simple & Efficient Attention Networks](https://mlsanity.com/paper/2207.05844/) (2022)
- [Motion Transformer with Global Intention Localization and Local Movement Refinement](https://mlsanity.com/paper/2209.13508/) (2022)
- [A Survey of Confidence Estimation and Calibration in Large Language Models](https://mlsanity.com/paper/2311.08298/) (2023)
- [MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusion](https://mlsanity.com/paper/2306.03083/) (2023)
- [MotionLM: Multi-Agent Motion Forecasting as Language Modeling](https://mlsanity.com/paper/2309.16534/) (2023)
- [MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Querying](https://mlsanity.com/paper/2306.17770/) (2023)
- [HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encoding](https://mlsanity.com/paper/2205.09753/) (2022)
- [M2I: From Factored Marginal Trajectory Prediction to Interactive Prediction](https://mlsanity.com/paper/2202.11884/) (2022)
- [Learning to Predict Vehicle Trajectories with Model-based Planning](https://mlsanity.com/paper/2103.04027/) (2021)
- [Solving Motion Planning Tasks with a Scalable Generative Model](https://mlsanity.com/paper/2407.02797/) (2024)
- [Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding](https://mlsanity.com/paper/2310.12970/) (2023)
- [Large Trajectory Models are Scalable Motion Predictors and Planners](https://mlsanity.com/paper/2310.19620/) (2023)
- [ControlMTR: Control-Guided Motion Transformer with Scene-Compliant Intention Points for Feasible Motion Prediction](https://mlsanity.com/paper/2404.10295/) (2024)
- [HYPER: Learned Hybrid Trajectory Prediction via Factored Inference and Adaptive Sampling](https://mlsanity.com/paper/2110.02344/) (2021)
- [MGTR: Multi-Granular Transformer for Motion Prediction with LiDAR](https://mlsanity.com/paper/2312.02409/) (2023)
- [Scaling Motion Forecasting Models with Ensemble Distillation](https://mlsanity.com/paper/2404.03843/) (2024)

## Built on by (most cited first, 29 of 29)

- [SimScale: Learning to Drive via Real-World Simulation at Scale](https://mlsanity.com/paper/2511.23369/) (2025)
- [Scaling Laws of Motion Forecasting and Planning -- Technical Report](https://mlsanity.com/paper/2506.08228/) (2025)
- [ReAL-AD: Towards Human-Like Reasoning in End-to-End Autonomous Driving](https://mlsanity.com/paper/2507.12499/) (2025)
- [Devil is in Narrow Policy: Unleashing Exploration in Driving VLA Models](https://mlsanity.com/paper/2603.06049/) (2026)
- [DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Driving](https://mlsanity.com/paper/2603.11041/) (2026)
- [Generative AI for Autonomous Driving: Frontiers and Opportunities](https://mlsanity.com/paper/2505.08854/) (2025)
- [ViLaD: A Large Vision Language Diffusion Framework for End-to-End Autonomous Driving](https://mlsanity.com/paper/2508.12603/) (2025)
- [Accelerating Structured Chain-of-Thought in Autonomous Vehicles](https://mlsanity.com/paper/2602.02864/) (2026)
- [StyleVLA: Driving Style-Aware Vision Language Action Model for Autonomous Driving](https://mlsanity.com/paper/2603.09482/) (2026)
- [CoT4AD: A Vision-Language-Action Model with Explicit Chain-of-Thought Reasoning for Autonomous Driving](https://mlsanity.com/paper/2511.22532/) (2025)
- [DAP: A Discrete-token Autoregressive Planner for Autonomous Driving](https://mlsanity.com/paper/2511.13306/) (2025)
- [NaviDriveVLM: Decoupling High-Level Reasoning and Motion Planning for Autonomous Driving](https://mlsanity.com/paper/2603.07901/) (2026)
- [MADrive: Memory-Augmented Driving Scene Modeling](https://mlsanity.com/paper/2506.21520/) (2025)
- [Efficient and Explainable End-to-End Autonomous Driving via Masked Vision-Language-Action Diffusion](https://mlsanity.com/paper/2602.20577/) (2026)
- [RDAR: Reward-Driven Agent Relevance Estimation for Autonomous Driving](https://mlsanity.com/paper/2509.19789/) (2025)
- [ETA-VLA: Efficient Token Adaptation via Temporal Fusion and Intra-LLM Sparsification for Vision-Language-Action Models](https://mlsanity.com/paper/2603.25766/) (2026)
- [VL-DPO: Vision-Language-Guided Finetuning for Preference-Aligned Autonomous Driving](https://mlsanity.com/paper/2605.20082/) (2026)
- [Motion Planning in Compressed Representation Spaces](https://mlsanity.com/paper/2606.30940/) (2026)
- [Conditional Flow Matching for Continuous Anomaly Detection in Autonomous Driving on a Manifold-Aware Spectral Space](https://mlsanity.com/paper/2602.17586/) (2026)
- [Toward Unified Multimodal Representation Learning for Autonomous Driving](https://mlsanity.com/paper/2603.07874/) (2026)
- [A Neuro-Symbolic Framework Combining Inductive and Deductive Reasoning for Autonomous Driving Planning](https://mlsanity.com/paper/2603.12421/) (2026)
- [Information Coordination as a Bridge: A Neuro-Symbolic Architecture for Reliable Autonomous Driving Scene Understanding](https://mlsanity.com/paper/2605.04475/) (2026)
- [STELLAR: Scaling 3D Perception Large Models for Autonomous Driving](https://mlsanity.com/paper/2605.20390/) (2026)
- [CityGen: Structure-Guided City-Style Synthesis for Cross-City Autonomous Driving](https://mlsanity.com/paper/2605.29935/) (2026)
- [Event-Adaptive Motion Planning with Distilled Vision-Language Model in Safety-Critical Situations](https://mlsanity.com/paper/2606.25629/) (2026)
- [Towards Metric-Agnostic Trajectory Forecasting](https://mlsanity.com/paper/2607.01133/) (2026)
- [Agent-driven Long-tail Simulation for Autonomous Driving](https://mlsanity.com/paper/2607.04331/) (2026)
- [GRAVA: Grounded Reasoning-to-Action Representation and Learning for Autonomous Driving](https://mlsanity.com/paper/2609.15169/) (2026)
- [Planning in the Backbone: DiffAdapterVLA for Native Continuous Trajectory Generation with Driving VLMs](https://mlsanity.com/paper/2609.15322/) (2026)

## Abstract (arXiv)

We present DriveGPT, a scalable behavior model for autonomous driving. We model driving as a sequential decision-making task, and learn a transformer model to predict future agent states as tokens in an autoregressive fashion. We scale up our model parameters and training data by multiple orders of magnitude, enabling us to explore the scaling properties in terms of dataset size, model parameters, and compute. We evaluate DriveGPT across different scales in a planning task, through both quantitative metrics and qualitative examples, including closed-loop driving in complex real-world scenarios. In a separate prediction task, DriveGPT outperforms state-of-the-art baselines and exhibits improved performance by pretraining on a large-scale dataset, further validating the benefits of data scaling.