mlsanityData to 14 Oct 2026

Energy-based Potential Games for Joint Motion Forecasting and Control

arXiv 2312.01811 ↗First posted 4 Dec 2023cs.LG cs.AI cs.GTcited by 9

Not on the map. Why?

Abstract (arXiv)

This work uses game theory as a mathematical framework to address interaction modeling in multi-agent motion forecasting and control. Despite its interpretability, applying game theory to real-world robotics, like automated driving, faces challenges such as unknown game parameters. To tackle these, we establish a connection between differential games, optimal control, and energy-based models, demonstrating how existing approaches can be unified under our proposed Energy-based Potential Game formulation. Building upon this, we introduce a new end-to-end learning application that combines neural networks for game-parameter inference with a differentiable game-theoretic optimization layer, acting as an inductive bias. The analysis provides empirical evidence that the game-theoretic layer adds interpretability and improves the predictive performance of various neural network backbones using two simulations and two real-world driving datasets.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 52 of its 75 references that are in the map. First posted on arXiv 4 Dec 2023.

Builds on

52 of 75 references.

References · 52

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  4. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
  5. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 599
  6. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  7. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 530
  8. 2021Implicit Behavioral Cloningcited by 452
  9. 2020TNT: Target-driveN Trajectory Predictioncited by 347
  10. 2021How to Train Your Energy-Based Modelscited by 274
  11. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  12. 2018Convolutional Social Pooling for Vehicle Trajectory Predictioncited by 221
38 more
  1. 2022Stochastic Trajectory Prediction via Motion Indeterminacy Diffusioncited by 185
  2. 2021DC3: A learning method for optimization with hard constraintscited by 184
  3. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 179
  4. 2019Multiple Futures Predictioncited by 178
  5. 2016A Theory of Generative ConvNetcited by 174
  6. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 168
  7. 2020Human Trajectory Forecasting in Crowds: A Deep Learning Perspectivecited by 129
  8. 2020Implicit Latent Variable Model for Scene-Consistent Motion Forecastingcited by 116
  9. 2021Reward (Mis)design for Autonomous Drivingcited by 104
  10. 2019What the Constant Velocity Model Can Teach Us About Pedestrian Motion Predictioncited by 80
  11. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  12. 2022Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Drivingcited by 73
  13. 2022Theseus: A Library for Differentiable Nonlinear Optimizationcited by 65
  14. 2020DSDNet: Deep Structured self-Driving Networkcited by 62
  15. 2016Cooperative Training of Descriptor and Generator Networkscited by 60
  16. 2022A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Modelcited by 44
  17. 2020Learning Energy-Based Model with Variational Auto-Encoder as Amortized Samplercited by 43
  18. 2022JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Drivingcited by 40
  19. 2019Efficient Iterative Linear-Quadratic Approximations for Nonlinear Multi-Player General-Sum Differential Gamescited by 37
  20. 2022DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehiclescited by 37
  21. 2017End-to-End Learning for Structured Prediction Energy Networkscited by 33
  22. 2019Model Based Planning with Energy Based Modelscited by 33
  23. 2021UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planningcited by 23
  24. 2021Contingencies from Observations: Tractable Contingency Planning with Learned Behavior Modelscited by 22
  25. 2021Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Spacecited by 20
  26. 2020LUCIDGames: Online Unscented Inverse Dynamic Games for Adaptive Trajectory Prediction and Planningcited by 17
  27. 2023Joint Metrics Matter: A Better Standard for Trajectory Forecastingcited by 17
  28. 2021ALGAMES: A Fast Augmented Lagrangian Solver for Constrained Dynamic Gamescited by 16
  29. 2019Energy-Based Continuous Inverse Optimal Controlcited by 14
  30. 2022Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computationcited by 14
  31. 2020Learning Game-Theoretic Models of Multiagent Trajectories Using Implicit Layerscited by 13
  32. 2021Inferring Objectives in Continuous Dynamic Games from Noise-Corrupted Partial State Observationscited by 13
  33. 2022Learning to Play Trajectory Games Against Opponents with Unknown Objectivescited by 11
  34. 2020Inference-Based Strategy Alignment for General-Sum Differential Gamescited by 10
  35. 2022Importance is in your attention: agent importance prediction for autonomous drivingcited by 10
  36. 2021Potential iLQR: A Potential-Minimizing Controller for Planning Multi-Agent Interactive Trajectoriescited by 8
  37. 2022Interventional Behavior Prediction: Avoiding Overly Confident Anticipation in Interactive Predictioncited by 8
  38. 2022Differentiable Constrained Imitation Learning for Robot Motion Planning and Controlcited by 7

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 2; 2026: 2.

9 papers.

  1. 2024DiffTORI: Differentiable Trajectory Optimization for Deep Reinforcement and Imitation Learningcited by 6
  2. 2024Blending Data-Driven Priors in Dynamic Gamescited by 6
  3. 2024LoRD: Adapting Differentiable Driving Policies to Distribution Shiftscited by 4
  4. 2024Learning Multiple Initial Solutions to Optimization Problemscited by 4
  5. 2024Auto-Encoding Bayesian Inverse Gamescited by 3
  6. 2025Recent Advances in Multi-Agent Human Trajectory Prediction: A Comprehensive Reviewcited by 2
  7. 2025Deep Fictitious Play-Based Potential Differential Games for Learning Human-Like Interaction at Unsignalized Intersectionscited by 0
  8. 2026Bayesian Inverse Games with High-Dimensional Multi-Modal Observationscited by 0
  9. 2026TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perceptioncited by 0