mlsanityData to 7 Oct 2026

Markov Potential Game and Multi-Agent Reinforcement Learning for Autonomous Driving

arXiv 2603.19188 ↗First posted 19 Mar 2026eess.SY cs.SYcited by 1

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Autonomous driving (AD) requires safe and reliable decision-making among interacting agents, e.g., vehicles, bicycles, and pedestrians. Multi-agent reinforcement learning (MARL) modeled by Markov games (MGs) provides a suitable framework to characterize such agents' interactions during decision-making. Nash equilibria (NEs) are often the desired solution in an MG. However, it is typically challenging to compute an NE in general-sum games, unless the game is a Markov potential game (MPG), which ensures the NE attainability under a few learning algorithms such as gradient play. However, it has been an open question how to construct an MPG and whether these construction rules are suitable for AD applications. In this paper, we provide sufficient conditions under which an MG is an MPG and show that these conditions can accommodate general driving objectives for autonomous vehicles (AVs) using highway forced merge scenarios as illustrative examples. A parameter-sharing neural network (NN) structure is designed to enable decentralized policy execution. The trained driving policy from MPGs is evaluated in both simulated and naturalistic traffic datasets. Comparative studies with single-agent RL and with human drivers whose behaviors are recorded in the traffic datasets are reported, respectively.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 20 of its 39 references that are in the map. First posted on arXiv 19 Mar 2026.

Builds on

20 of 39 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey19 Aug
  2. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learning6 Jan

Other references · 18

  1. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  2. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  3. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  4. 2020Near-Optimal Reinforcement Learning with Self-Playcited by 108
  5. 2021Global Convergence of Multi-Agent Policy Gradient in Markov Potential Gamescited by 89
  6. 2021Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharingcited by 85
  7. 2020No-regret learning and mixed Nash equilibria: They do not mixcited by 49
  8. 2021Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Trafficcited by 38
  9. 2021Gradient play in stochastic games: stationary points, convergence, and sample complexitycited by 32
  10. 2016Game-Theoretic Modeling of Driver and Vehicle Interactions for Verification and Validation of Autonomous Vehicle Control Systemscited by 24
  11. 2020An Integrated Framework of Decision Making and Motion Planning for Autonomous Vehicles Considering Social Behaviorscited by 16
  12. 2021GamePlan: Game-Theoretic Multi-Agent Planning with Human Drivers at Intersections, Roundabouts, and Mergingcited by 16
6 more
  1. 2023Dealing With Non-stationarity in Decentralized Cooperative Multi-Agent Deep Reinforcement Learning via Multi-Timescale Learningcited by 12
  2. 2022Potential Game-Based Decision-Making for Autonomous Drivingcited by 9
  3. 2022For Learning in Symmetric Teams, Local Optima are Global Nash Equilibriacited by 6
  4. 2020Towards a Systematic Computational Framework for Modeling Multi-Agent Decision-Making at Micro Level for Smart Vehicles in a Smart Worldcited by 3
  5. 2022Safe and Human-Like Autonomous Driving: A Predictor-Corrector Potential Game Approachcited by 3
  6. 2023Game Projection and Robustness for Game-Theoretic Autonomous Drivingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Meta-Multi-Agent Reinforcement Learning for Fast Adaptation of Interactive Policies with Applications to Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Markov Potential Game Construction and Multi-Agent Reinforcement Learning with Applications to Autonomous Driving ↗
  2. 2021Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic
  3. 2022Discrete Control in Real-World Driving Environments using Deep Reinforcement Learning
  4. 2025MARL-OT: Multi-Agent Reinforcement Learning Guided Online Fuzzing to Detect Safety Violation in Autonomous Driving Systems
  5. 2025Hybrid Action Based Reinforcement Learning for Multi-Objective Compatible Autonomous Driving
  6. 2023Non-zero-sum Game Control for Multi-vehicle Driving via Reinforcement Learning
  7. 2021Reinforcement Learning Based Safe Decision Making for Highway Autonomous Driving
  8. 2024Integrated Decision Making and Trajectory Planning for Autonomous Driving Under Multimodal Uncertainties: A Bayesian Game Approach