Solving Motion Planning Tasks with a Scalable Generative Model
Not on the map. Why?
Abstract (arXiv)
As autonomous driving systems being deployed to millions of vehicles, there is a pressing need of improving the system's scalability, safety and reducing the engineering cost. A realistic, scalable, and practical simulator of the driving world is highly desired. In this paper, we present an efficient solution based on generative models which learns the dynamics of the driving scenes. With this model, we can not only simulate the diverse futures of a given driving scenario but also generate a variety of driving scenarios conditioned on various prompts. Our innovative design allows the model to operate in both full-Autoregressive and partial-Autoregressive modes, significantly improving inference and training speed without sacrificing generative capability. This efficiency makes it ideal for being used as an online reactive environment for reinforcement learning, an evaluator for planning policies, and a high-fidelity simulator for testing. We evaluated our model against two real-world datasets: the Waymo motion dataset and the nuPlan dataset. On the simulation realism and scene generation benchmark, our model achieves the state-of-the-art performance. And in the planning benchmarks, our planner outperforms the prior arts. We conclude that the proposed generative model may serve as a foundation for a variety of motion planning tasks, including data generation, simulation, planning, and online training. Source code is public at https://github.com/HorizonRobotics/GUMP/
Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 64 of its 82 references that are in the map. First posted on arXiv 3 Jul 2024.
Builds on
64 of 82 references.
References · 64
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
- 2023Mastering Diverse Domains through World Modelscited by 1,193
38 more
- 2018Hierarchical Neural Story Generationcited by 1,182
- 2022MaskGIT: Masked Generative Image Transformercited by 1,082
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2020Mastering Atari with Discrete World Modelscited by 990
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
- 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
- 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
- 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
- 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
- 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 311
- 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
- 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
- 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 279
- 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 267
- 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
- 2022VectorMapNet: End-to-end Vectorized HD Map Learningcited by 243
- 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
- 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
- 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 192
- 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
- 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 154
- 2022Model-Based Imitation Learning for Urban Drivingcited by 149
- 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
- 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 139
- 2019Meta-Sim: Learning to Generate Synthetic Datasetscited by 128
- 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
- 2021Learning to drive from a world on railscited by 114
- 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
- 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusioncited by 105
- 2022TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarioscited by 104
- 2018Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Datacited by 94
- 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 92
- 2023The Waymo Open Sim Agents Challengecited by 86
- 2021SceneGen: Learning to Generate Realistic Traffic Scenescited by 83
- 2023DrivingDiffusion: Layout-Guided multi-view driving scene video generation with latent diffusion modelcited by 75
- 2022When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learningcited by 59
50 most cited of 64 All 64 (JSON)
Built on by
Citing papers per year: 2024: 7; 2025: 23; 2026: 10.
40 papers.
Sort:
By area:
- 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 138
- 2025RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learningcited by 76
- 2025DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Drivingcited by 52
- 2024Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuningcited by 41
- 2024Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Modelscited by 40
- 2024DriveGPT: Scaling Autoregressive Behavior Models for Drivingcited by 29
- 2025Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environmentscited by 27
- 2024Generalizing Motion Planners with Mixture of Experts for Autonomous Drivingcited by 22
- 2025Plan-R1: Safe and Feasible Trajectory Planning as Language Modelingcited by 19
- 2024KiGRAS: Kinematic-Driven Generative Model for Realistic Agent Simulationcited by 17
- 2025DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Drivingcited by 17
- 2025Data Scaling Laws for End-to-End Autonomous Drivingcited by 14
28 more
- 2025Advancing Multi-agent Traffic Simulation via R1-Style Reinforcement Fine-Tuningcited by 14
- 2025UniMM: A Unified Mixture Model Framework for Multi-Agent Simulationcited by 13
- 2025Generating Multimodal Driving Scenes via Next-Scene Predictioncited by 12
- 2025Decoupled Diffusion Sparks Adaptive Scene Generationcited by 12
- 2025DRoPE: Directional Rotary Position Embedding for Efficient Agent Interaction Modelingcited by 9
- 2024Learning Multiple Probabilistic Decisions from Latent World Model in Autonomous Drivingcited by 8
- 2025When Planners Meet Reality: How Learned, Reactive Traffic Agents Shift nuPlan Benchmarkscited by 7
- 2025VLMPlanner: Integrating Visual Language Models with Motion Planningcited by 6
- 2025The Safety Challenge of World Models for Embodied AI Agents: A Reviewcited by 5
- 2025DecompGAIL: Learning Realistic Traffic Behaviors with Decomposed Multi-Agent Generative Adversarial Imitation Learningcited by 5
- 2025RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulationcited by 4
- 2025Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Drivingcited by 4
- 2025Autoregressive Meta-Actions for Unified Controllable Trajectory Generationcited by 2
- 2025Long-term Traffic Simulation with Interleaved Autoregressive Motion and Scenario Generationcited by 2
- 2025SceneStreamer: Continuous Scenario Generation as Next Token Group Predictioncited by 2
- 2025Optimization-Guided Diffusion for Interactive Scene Generationcited by 2
- 2026CorrectionPlanner: Self-Correction Planner with Reinforcement Learning in Autonomous Drivingcited by 2
- 2026RLFTSim: Realistic and Controllable Multi-Agent Traffic Simulation via Reinforcement Learning Fine-Tuningcited by 2
- 2024TrafficGamer: Reliable and Flexible Traffic Simulation for Safety-Critical Scenarios with Game-Theoretic Oraclescited by 1
- 2026SaFeR: Safety-Critical Scenario Generation for Autonomous Driving Test via Feasibility-Constrained Token Resamplingcited by 1
- 2026Learning Rollout from Sampling:An R1-Style Tokenized Traffic Simulation Modelcited by 1
- 2026ASSCG: Just-Right Gating over Chattering for Fast-Slow LLM Planning in Autonomous Drivingcited by 1
- 2026World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planningcited by 1
- 2026DiffuSearch: How Hybrid Trajectory Planning Benefits from Aligned Objectives in Diffusion and Action Spacecited by 1
- 2025Pseudo-Expert Regularized Offline RL for End-to-End Autonomous Driving in Photorealistic Closed-Loop Environmentscited by 0
- 2026Bridging Local Observation and Global Simulation in Closed-Loop Traffic Modelingcited by 0
- 2026Controllable Sim Agents with Behavior Latentscited by 0
- 2026Agent-driven Long-tail Simulation for Autonomous Drivingcited by 0
- 2026DiffuSearch: How Hybrid Trajectory Planning Benefits from Aligned Objectives in Diffusion and Action Space2 Sep 2026
- 2026World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning12 Jul 2026
- 2026Agent-driven Long-tail Simulation for Autonomous Driving5 Jul 2026
- 2026Controllable Sim Agents with Behavior Latents2 Jul 2026
- 2026Bridging Local Observation and Global Simulation in Closed-Loop Traffic Modeling30 Jun 2026
- 2026ASSCG: Just-Right Gating over Chattering for Fast-Slow LLM Planning in Autonomous Driving24 Jun 2026
- 2026RLFTSim: Realistic and Controllable Multi-Agent Traffic Simulation via Reinforcement Learning Fine-Tuning18 May 2026
- 2026Learning Rollout from Sampling:An R1-Style Tokenized Traffic Simulation Model26 Mar 2026
- 2026CorrectionPlanner: Self-Correction Planner with Reinforcement Learning in Autonomous Driving16 Mar 2026
- 2026SaFeR: Safety-Critical Scenario Generation for Autonomous Driving Test via Feasibility-Constrained Token Resampling4 Mar 2026
- 2025Pseudo-Expert Regularized Offline RL for End-to-End Autonomous Driving in Photorealistic Closed-Loop Environments21 Dec 2025
- 2025DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Driving8 Dec 2025
28 more
- 2025Optimization-Guided Diffusion for Interactive Scene Generation8 Dec 2025
- 2025When Planners Meet Reality: How Learned, Reactive Traffic Agents Shift nuPlan Benchmarks16 Oct 2025
- 2025DecompGAIL: Learning Realistic Traffic Behaviors with Decomposed Multi-Agent Generative Adversarial Imitation Learning8 Oct 2025
- 2025The Safety Challenge of World Models for Embodied AI Agents: A Review7 Oct 2025
- 2025Advancing Multi-agent Traffic Simulation via R1-Style Reinforcement Fine-Tuning28 Sep 2025
- 2025Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Driving2 Sep 2025
- 2025VLMPlanner: Integrating Visual Language Models with Motion Planning27 Jul 2025
- 2025DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Driving5 Jul 2025
- 2025SceneStreamer: Continuous Scenario Generation as Next Token Group Prediction29 Jun 2025
- 2025Long-term Traffic Simulation with Interleaved Autoregressive Motion and Scenario Generation20 Jun 2025
- 2025Autoregressive Meta-Actions for Unified Controllable Trajectory Generation29 May 2025
- 2025Plan-R1: Safe and Feasible Trajectory Planning as Language Modeling23 May 2025
- 2025RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulation6 May 2025
- 2025Decoupled Diffusion Sparks Adaptive Scene Generation14 Apr 2025
- 2025Data Scaling Laws for End-to-End Autonomous Driving6 Apr 2025
- 2025Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environments28 Mar 2025
- 2025DRoPE: Directional Rotary Position Embedding for Efficient Agent Interaction Modeling19 Mar 2025
- 2025Generating Multimodal Driving Scenes via Next-Scene Prediction19 Mar 2025
- 2025RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learning18 Feb 2025
- 2025UniMM: A Unified Mixture Model Framework for Multi-Agent Simulation28 Jan 2025
- 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidance26 Jan 2025
- 2024DriveGPT: Scaling Autoregressive Behavior Models for Driving19 Dec 2024
- 2024Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Models5 Dec 2024
- 2024Generalizing Motion Planners with Mixture of Experts for Autonomous Driving21 Oct 2024
- 2024Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuning8 Oct 2024
- 2024Learning Multiple Probabilistic Decisions from Latent World Model in Autonomous Driving24 Sep 2024
- 2024TrafficGamer: Reliable and Flexible Traffic Simulation for Safety-Critical Scenarios with Game-Theoretic Oracles28 Aug 2024
- 2024KiGRAS: Kinematic-Driven Generative Model for Realistic Agent Simulation17 Jul 2024
- 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 138
- 2025RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learningcited by 76
- 2024DriveGPT: Scaling Autoregressive Behavior Models for Drivingcited by 29
- 2024Generalizing Motion Planners with Mixture of Experts for Autonomous Drivingcited by 22
- 2025Data Scaling Laws for End-to-End Autonomous Drivingcited by 14
- 2025Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Drivingcited by 4
- 2026CorrectionPlanner: Self-Correction Planner with Reinforcement Learning in Autonomous Drivingcited by 2
- 2024TrafficGamer: Reliable and Flexible Traffic Simulation for Safety-Critical Scenarios with Game-Theoretic Oraclescited by 1
- 2026SaFeR: Safety-Critical Scenario Generation for Autonomous Driving Test via Feasibility-Constrained Token Resamplingcited by 1
- 2026ASSCG: Just-Right Gating over Chattering for Fast-Slow LLM Planning in Autonomous Drivingcited by 1
- 2025Pseudo-Expert Regularized Offline RL for End-to-End Autonomous Driving in Photorealistic Closed-Loop Environmentscited by 0
- 2026Agent-driven Long-tail Simulation for Autonomous Drivingcited by 0
- 2026Agent-driven Long-tail Simulation for Autonomous Driving5 Jul 2026
- 2026ASSCG: Just-Right Gating over Chattering for Fast-Slow LLM Planning in Autonomous Driving24 Jun 2026
- 2026CorrectionPlanner: Self-Correction Planner with Reinforcement Learning in Autonomous Driving16 Mar 2026
- 2026SaFeR: Safety-Critical Scenario Generation for Autonomous Driving Test via Feasibility-Constrained Token Resampling4 Mar 2026
- 2025Pseudo-Expert Regularized Offline RL for End-to-End Autonomous Driving in Photorealistic Closed-Loop Environments21 Dec 2025
- 2025Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Driving2 Sep 2025
- 2025Data Scaling Laws for End-to-End Autonomous Driving6 Apr 2025
- 2025RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learning18 Feb 2025
- 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidance26 Jan 2025
- 2024DriveGPT: Scaling Autoregressive Behavior Models for Driving19 Dec 2024
- 2024Generalizing Motion Planners with Mixture of Experts for Autonomous Driving21 Oct 2024
- 2024TrafficGamer: Reliable and Flexible Traffic Simulation for Safety-Critical Scenarios with Game-Theoretic Oracles28 Aug 2024
- 2025DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Drivingcited by 52
- 2024Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuningcited by 41
- 2025Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environmentscited by 27
- 2025DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Drivingcited by 17
- 2025Optimization-Guided Diffusion for Interactive Scene Generationcited by 2
- 2026DiffuSearch: How Hybrid Trajectory Planning Benefits from Aligned Objectives in Diffusion and Action Spacecited by 1
- 2026DiffuSearch: How Hybrid Trajectory Planning Benefits from Aligned Objectives in Diffusion and Action Space2 Sep 2026
- 2025DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Driving8 Dec 2025
- 2025Optimization-Guided Diffusion for Interactive Scene Generation8 Dec 2025
- 2025DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Driving5 Jul 2025
- 2025Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environments28 Mar 2025
- 2024Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuning8 Oct 2024
- 2024Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Modelscited by 40
- 2024KiGRAS: Kinematic-Driven Generative Model for Realistic Agent Simulationcited by 17
- 2025UniMM: A Unified Mixture Model Framework for Multi-Agent Simulationcited by 13
- 2025Generating Multimodal Driving Scenes via Next-Scene Predictioncited by 12
- 2025Decoupled Diffusion Sparks Adaptive Scene Generationcited by 12
- 2025DRoPE: Directional Rotary Position Embedding for Efficient Agent Interaction Modelingcited by 9
- 2025When Planners Meet Reality: How Learned, Reactive Traffic Agents Shift nuPlan Benchmarkscited by 7
- 2025RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulationcited by 4
- 2025Autoregressive Meta-Actions for Unified Controllable Trajectory Generationcited by 2
- 2025Long-term Traffic Simulation with Interleaved Autoregressive Motion and Scenario Generationcited by 2
- 2025SceneStreamer: Continuous Scenario Generation as Next Token Group Predictioncited by 2
- 2026RLFTSim: Realistic and Controllable Multi-Agent Traffic Simulation via Reinforcement Learning Fine-Tuningcited by 2
12 most cited of 15 All 15 (JSON)
- 2026Controllable Sim Agents with Behavior Latents2 Jul 2026
- 2026Bridging Local Observation and Global Simulation in Closed-Loop Traffic Modeling30 Jun 2026
- 2026RLFTSim: Realistic and Controllable Multi-Agent Traffic Simulation via Reinforcement Learning Fine-Tuning18 May 2026
- 2026Learning Rollout from Sampling:An R1-Style Tokenized Traffic Simulation Model26 Mar 2026
- 2025When Planners Meet Reality: How Learned, Reactive Traffic Agents Shift nuPlan Benchmarks16 Oct 2025
- 2025SceneStreamer: Continuous Scenario Generation as Next Token Group Prediction29 Jun 2025
- 2025Long-term Traffic Simulation with Interleaved Autoregressive Motion and Scenario Generation20 Jun 2025
- 2025Autoregressive Meta-Actions for Unified Controllable Trajectory Generation29 May 2025
- 2025RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulation6 May 2025
- 2025Decoupled Diffusion Sparks Adaptive Scene Generation14 Apr 2025
- 2025DRoPE: Directional Rotary Position Embedding for Efficient Agent Interaction Modeling19 Mar 2025
- 2025Generating Multimodal Driving Scenes via Next-Scene Prediction19 Mar 2025
12 newest of 15 All 15 (JSON)