# Multi-Agent Reinforcement Learning for Markov Routing Games: A New Modeling Paradigm For Dynamic Traffic Assignment

arXiv: https://arxiv.org/abs/2011.10915 · HTML: https://mlsanity.com/paper/2011.10915/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2020.

## Built on by (most cited first, 9 of 9)

- [AI-Driven Day-to-Day Route Choice](https://mlsanity.com/paper/2412.03338/) (2024)
- [CVLight: Decentralized Learning for Adaptive Traffic Signal Control with Connected Vehicles](https://mlsanity.com/paper/2104.10340/) (2021)
- [URB -- Urban Routing Benchmark for RL-equipped Connected Autonomous Vehicles](https://mlsanity.com/paper/2505.17734/) (2025)
- [A Single Online Agent Can Efficiently Learn Mean Field Games](https://mlsanity.com/paper/2405.03718/) (2024)
- [Reinforcement Learning-based Sequential Route Recommendation for System-Optimal Traffic Assignment](https://mlsanity.com/paper/2505.20889/) (2025)
- [Scalable and Reliable Multi-agent Reinforcement Learning for Traffic Assignment](https://mlsanity.com/paper/2506.17029/) (2025)
- [Federated Hierarchical Reinforcement Learning for Adaptive Traffic Signal Control](https://mlsanity.com/paper/2504.05553/) (2025)
- [EcoFair-CH-MARL: Scalable Constrained Hierarchical Multi-Agent RL with Real-Time Emission Budgets and Fairness Guarantees](https://mlsanity.com/paper/2603.14625/) (2026)
- [Cooperative Path Planning with Asynchronous Multiagent Reinforcement Learning](https://mlsanity.com/paper/2409.00754/) (2024)

## Abstract (arXiv)

This paper aims to develop a paradigm that models the learning behavior of intelligent agents (including but not limited to autonomous vehicles, connected and automated vehicles, or human-driven vehicles with intelligent navigation systems where human drivers follow the navigation instructions completely) with a utility-optimizing goal and the system's equilibrating processes in a routing game among atomic selfish agents. Such a paradigm can assist policymakers in devising optimal operational and planning countermeasures under both normal and abnormal circumstances. To this end, we develop a Markov routing game (MRG) in which each agent learns and updates her own en-route path choice policy while interacting with others in transportation networks. To efficiently solve MRG, we formulate it as multi-agent reinforcement learning (MARL) and devise a mean field multi-agent deep Q learning (MF-MA-DQL) approach that captures the competition among agents. The linkage between the classical DUE paradigm and our proposed Markov routing game (MRG) is discussed. We show that the routing behavior of intelligent agents is shown to converge to the classical notion of predictive dynamic user equilibrium (DUE) when traffic environments are simulated using dynamic loading models (DNL). In other words, the MRG depicts DUEs assuming perfect information and deterministic environments propagated by DNL models. Four examples are solved to illustrate the algorithm efficiency and consistency between DUE and the MRG equilibrium, on a simple network without and with spillback, the Ortuzar Willumsen (OW) Network, and a real-world network near Columbia University's campus in Manhattan of New York City.