# BEVGPT: Generative Pre-trained Large Model for Autonomous Driving Prediction, Decision-Making, and Planning

arXiv: https://arxiv.org/abs/2310.10357 · HTML: https://mlsanity.com/paper/2310.10357/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2023.

## Built on by (most cited first, 15 of 15)

- [Vision Language Models in Autonomous Driving: A Survey and Outlook](https://mlsanity.com/paper/2310.14414/) (2023)
- [DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Driving](https://mlsanity.com/paper/2405.04390/) (2024)
- [VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervision](https://mlsanity.com/paper/2412.14446/) (2024)
- [OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Driving](https://mlsanity.com/paper/2405.20337/) (2024)
- [Making Large Language Models Better Planners with Reasoning-Decision Alignment](https://mlsanity.com/paper/2408.13890/) (2024)
- [ChatSUMO: Large Language Model for Automating Traffic Scenario Generation in Simulation of Urban MObility](https://mlsanity.com/paper/2409.09040/) (2024)
- [RoboTron-Drive: All-in-One Large Multimodal Model for Autonomous Driving](https://mlsanity.com/paper/2412.07689/) (2024)
- [Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectives](https://mlsanity.com/paper/2402.02968/) (2024)
- [PKRD-CoT: A Unified Chain-of-thought Prompting for Multi-Modal Large Language Models in Autonomous Driving](https://mlsanity.com/paper/2412.02025/) (2024)
- [Applications of Large Scale Foundation Models for Autonomous Driving](https://mlsanity.com/paper/2311.12144/) (2023)
- [MoST: Multi-modality Scene Tokenization for Motion Prediction](https://mlsanity.com/paper/2404.19531/) (2024)
- [Prospective Role of Foundation Models in Advancing Autonomous Vehicles](https://mlsanity.com/paper/2405.02288/) (2023)
- [Pre-trained Transformer-Enabled Strategies with Human-Guided Fine-Tuning for End-to-end Navigation of Autonomous Vehicles](https://mlsanity.com/paper/2402.12666/) (2024)
- [Opportunistic Collaborative Planning with Large Vision Model Guided Control and Joint Query-Service Optimization](https://mlsanity.com/paper/2504.18057/) (2025)
- [Towards Physics-informed Spatial Intelligence with Human Priors: An Autonomous Driving Pilot Study](https://mlsanity.com/paper/2510.21160/) (2025)

## Abstract (arXiv)

Prediction, decision-making, and motion planning are essential for autonomous driving. In most contemporary works, they are considered as individual modules or combined into a multi-task learning paradigm with a shared backbone but separate task heads. However, we argue that they should be integrated into a comprehensive framework. Although several recent approaches follow this scheme, they suffer from complicated input representations and redundant framework designs. More importantly, they can not make long-term predictions about future driving scenarios. To address these issues, we rethink the necessity of each module in an autonomous driving task and incorporate only the required modules into a minimalist autonomous driving framework. We propose BEVGPT, a generative pre-trained large model that integrates driving scenario prediction, decision-making, and motion planning. The model takes the bird's-eye-view (BEV) images as the only input source and makes driving decisions based on surrounding traffic scenarios. To ensure driving trajectory feasibility and smoothness, we develop an optimization-based motion planning method. We instantiate BEVGPT on Lyft Level 5 Dataset and use Woven Planet L5Kit for realistic driving simulation. The effectiveness and robustness of the proposed framework are verified by the fact that it outperforms previous methods in 100% decision-making metrics and 66% motion planning metrics. Furthermore, the ability of our framework to accurately generate BEV images over the long term is demonstrated through the task of driving scenario prediction. To the best of our knowledge, this is the first generative pre-trained large model for autonomous driving prediction, decision-making, and motion planning with only BEV images as input.