mlsanityData to 14 Oct 2026

GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving

arXiv 2303.05760 ↗First posted 10 Mar 2023cs.ROcited by 141

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

Abstract (arXiv)

Autonomous vehicles operating in complex real-world environments require accurate predictions of interactive behaviors between traffic participants. This paper tackles the interaction prediction problem by formulating it with hierarchical game theory and proposing the GameFormer model for its implementation. The model incorporates a Transformer encoder, which effectively models the relationships between scene elements, alongside a novel hierarchical Transformer decoder structure. At each decoding level, the decoder utilizes the prediction outcomes from the previous level, in addition to the shared environmental context, to iteratively refine the interaction process. Moreover, we propose a learning process that regulates an agent's behavior at the current level to respond to other agents' behaviors from the preceding level. Through comprehensive experiments on large-scale real-world driving datasets, we demonstrate the state-of-the-art accuracy of our model on the Waymo interaction prediction task. Additionally, we validate the model's capacity to jointly reason about the motion plan of the ego agent and the behaviors of multiple agents in both open-loop and closed-loop planning tests, outperforming various baseline methods. Furthermore, we evaluate the efficacy of our model on the nuPlan planning benchmark, where it achieves leading performance.

Key facts. Cited by 141 papers in the map (data through 14 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 40 of its 51 references that are in the map. First posted on arXiv 10 Mar 2023.

Builds on

40 of 51 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Driving17 Dec
  2. 2022Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Driving21 Jul
  3. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients28 Apr

Other references · 37

  1. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  2. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  3. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 530
  4. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 493
  5. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 397
  6. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 331
  7. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 295
  8. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  9. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 280
  10. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 270
  11. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  12. 2021End-to-end Interpretable Neural Motion Plannercited by 242
25 more
  1. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 212
  2. 2021GOHOME: Graph-Oriented Heatmap Output for future Motion Estimationcited by 146
  3. 2020Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?cited by 139
  4. 2021HOME: Heatmap Output for future Motion Estimationcited by 122
  5. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 110
  6. 2021THOMAS: Trajectory Heatmap Output with learned Multi-Agent Samplingcited by 101
  7. 2021LookOut: Diverse Multi-Future Prediction and Planning for Self-Drivingcited by 96
  8. 2022M2I: From Factored Marginal Trajectory Prediction to Interactive Predictioncited by 81
  9. 2020PiP: Planning-informed Trajectory Prediction for Autonomous Drivingcited by 80
  10. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  11. 2018Deep Imitative Models for Flexible Inference, Planning, and Controlcited by 76
  12. 2021Multi-modal Motion Prediction with Transformer-based Neural Network for Autonomous Drivingcited by 73
  13. 2022ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planningcited by 64
  14. 2021Identifying Driver Interactions via Conditional Behavior Predictioncited by 48
  15. 2020Multi-modal Sensor Fusion-Based Deep Neural Network for End-to-end Autonomous Driving with Scene Understandingcited by 42
  16. 2022Deep Interactive Motion Prediction and Planning: Playing Games with Motion Prediction Modelscited by 27
  17. 2022Social Interactions for Autonomous Driving: A Review and Perspectivescited by 27
  18. 2022InterSim: Interactive Traffic Simulation via Explicit Relation Modelingcited by 27
  19. 2016Game-Theoretic Modeling of Driver and Vehicle Interactions for Verification and Validation of Autonomous Vehicle Control Systemscited by 24
  20. 2020IDE-Net: Interactive Driving Event and Pattern Extraction from Human Datacited by 19
  21. 2021Deep Structured Reactive Planningcited by 19
  22. 2022MPA: MultiPath++ Based Architecture for Motion Predictioncited by 13
  23. 2021Cooperative Decision Making of Connected Automated Vehicles at Multi-lane Merging Zone: A Coalitional Game Approachcited by 10
  24. 2021AIR2 for Interaction Predictioncited by 9
  25. 2022ReCoAt: A Deep Learning-based Framework for Multi-Modal Motion Prediction in Autonomous Driving Applicationcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 15; 2024: 41; 2025: 42; 2026: 43.

141 papers.

Sort:

By area:

  1. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 260
  2. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 242
  3. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 233
  4. 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 141
  5. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 126
  6. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 104
  7. 2024DRAMA: An Efficient End-to-end Motion Planner for Autonomous Driving with Mambacited by 87
  8. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Drivingcited by 84
  9. 2023LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoningcited by 59
  10. 2023DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Drivingcited by 46
  11. 2024RenderWorld: World Model with Self-Supervised 3D Labelcited by 46
  12. 2024Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuningcited by 42
38 more
  1. 2024Solving Motion Planning Tasks with a Scalable Generative Modelcited by 40
  2. 2024Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios?cited by 38
  3. 2024PlanAgent: A Multi-modal Large Language Agent for Closed-loop Vehicle Motion Planningcited by 38
  4. 2024Asynchronous Large Language Model Enhanced Planner for Autonomous Drivingcited by 38
  5. 2025CarPlanner: Consistent Auto-regressive Trajectory Planning for Large-scale Reinforcement Learning in Autonomous Drivingcited by 34
  6. 2023PPAD: Iterative Interactions of Prediction and Planning for End-to-end Autonomous Drivingcited by 33
  7. 2024GaussianAD: Gaussian-Centric End-to-End Autonomous Drivingcited by 31
  8. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  9. 2023Rethinking Imitation-based Planner for Autonomous Drivingcited by 25
  10. 2024SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Predictioncited by 25
  11. 2024Generalizing Motion Planners with Mixture of Experts for Autonomous Drivingcited by 22
  12. 2025Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modelingcited by 22
  13. 2025Plan-R1: Safe and Feasible Trajectory Planning as Language Modelingcited by 20
  14. 2025Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Drivingcited by 19
  15. 2024Hybrid-Prediction Integrated Planning for Autonomous Drivingcited by 18
  16. 2025DriveAction: A Benchmark for Exploring Human-like Driving Decisions in VLA Modelscited by 17
  17. 2026ResWorld: Temporal Residual World Model for End-to-End Autonomous Drivingcited by 15
  18. 2025Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directionscited by 14
  19. 2024AMP: Autoregressive Motion Prediction Revisited with Next Token Prediction for Autonomous Drivingcited by 13
  20. 2024CALMM-Drive: Confidence-Aware Autonomous Driving with Large Multimodal Modelcited by 12
  21. 2025OccProphet: Pushing Efficiency Frontier of Camera-Only 4D Occupancy Forecasting with Observer-Forecaster-Refiner Frameworkcited by 12
  22. 2025Decoupled Diffusion Sparks Adaptive Scene Generationcited by 12
  23. 2024SemanticFormer: Holistic and Semantic Traffic Scene Representation for Trajectory Prediction using Knowledge Graphscited by 11
  24. 2024Instruct Large Language Models to Drive like Humanscited by 11
  25. 2024Planning with Adaptive World Models for Autonomous Drivingcited by 11
  26. 2025Dream to Drive with Predictive Individual World Modelcited by 11
  27. 2023Deception Game: Closing the Safety-Learning Loop in Interactive Robot Autonomycited by 10
  28. 2023Categorical Traffic Transformer: Interpretable and Diverse Behavior Prediction with Tokenized Latentcited by 10
  29. 2023Social Motion Prediction with Cognitive Hierarchiescited by 9
  30. 2025Research Challenges and Progress in the End-to-End V2X Cooperative Autonomous Driving Competitioncited by 9
  31. 2026nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Drivingcited by 9
  32. 2023Occupancy Prediction-Guided Neural Planner for Autonomous Drivingcited by 8
  33. 2024DaCapo: Accelerating Continuous Learning in Autonomous Systems for Video Analyticscited by 8
  34. 2024Learning Multiple Probabilistic Decisions from Latent World Model in Autonomous Drivingcited by 8
  35. 2025Motion Forecasting for Autonomous Vehicles: A Surveycited by 8
  36. 2025TrajFlow: Multi-modal Motion Prediction via Flow Matchingcited by 8
  37. 2024JointMotion: Joint Self-Supervision for Joint Motion Predictioncited by 7
  38. 2024Approaching Outside: Scaling Unsupervised 3D Object Detection from 2D Scenecited by 7

50 most cited of 141 All 141 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectories
  2. 2023Learning Interaction-aware Motion Prediction Model for Decision-making in Autonomous Driving
  3. 2022Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Driving
  4. 2025Driving is a Game: Combining Planning and Prediction with Bayesian Iterative Best Response
  5. 2025MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
  6. 2026ReWAM: Reciprocal World Action Models for Interactive Autonomous Driving
  7. 2023TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction
  8. 2026INTERACT: Interactive Planning for Autonomous Driving via Anchor-Conditioned Prediction and Trust-Region Refinement