# Reinforcement Learning-based Sequential Route Recommendation for System-Optimal Traffic Assignment

arXiv: https://arxiv.org/abs/2505.20889 · HTML: https://mlsanity.com/paper/2505.20889/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 52 references that are in the map.
- First posted on arXiv 27 May 2025.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 12 of 12)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control](https://mlsanity.com/paper/1903.04527/) (2019)
- [End-to-End Heterogeneous Graph Neural Networks for Traffic Assignment](https://mlsanity.com/paper/2310.13193/) (2023)
- [AI-Driven Day-to-Day Route Choice](https://mlsanity.com/paper/2412.03338/) (2024)
- [Multi-Agent Reinforcement Learning for Markov Routing Games: A New Modeling Paradigm For Dynamic Traffic Assignment](https://mlsanity.com/paper/2011.10915/) (2020)
- [Real-time system optimal traffic routing under uncertainties -- Can physics models boost reinforcement learning?](https://mlsanity.com/paper/2407.07364/) (2024)
- [Model-based graph reinforcement learning for inductive traffic signal control](https://mlsanity.com/paper/2208.00659/) (2022)
- [A Survey of Route Recommendations: Methods, Applications, and Opportunities](https://mlsanity.com/paper/2403.00284/) (2024)
- [The Braess Paradox in Dynamic Traffic](https://mlsanity.com/paper/2203.03726/) (2022)
- [Route Recommendations for Traffic Management Under Learned Partial Driver Compliance](https://mlsanity.com/paper/2504.02993/) (2025)

## Built on by (most cited first, 2 of 2)

- [Scalable and Reliable Multi-agent Reinforcement Learning for Traffic Assignment](https://mlsanity.com/paper/2506.17029/) (2025)
- [An LLM-Powered Cooperative Framework for Large-Scale Multi-Vehicle Navigation](https://mlsanity.com/paper/2510.07825/) (2025)

## Abstract (arXiv)

Modern navigation systems and shared mobility platforms increasingly rely on personalized route recommendations to improve individual travel experience and operational efficiency. However, a key question remains: can such sequential, personalized routing decisions collectively lead to system-optimal (SO) traffic assignment? This paper addresses this question by proposing a learning-based framework that reformulates the static SO traffic assignment problem as a single-agent deep reinforcement learning (RL) task. A central agent sequentially recommends routes to travelers as origin-destination (OD) demands arrive, to minimize total system travel time. To enhance learning efficiency and solution quality, we develop an MSA-guided deep Q-learning algorithm that integrates the iterative structure of traditional traffic assignment methods into the RL training process. The proposed approach is evaluated on both the Braess and Ortuzar-Willumsen (OW) networks. Results show that the RL agent converges to the theoretical SO solution in the Braess network and achieves only a 0.35% deviation in the OW network. Further ablation studies demonstrate that the route action set's design significantly impacts convergence speed and final performance, with SO-informed route sets leading to faster learning and better outcomes. This work provides a theoretically grounded and practically relevant approach to bridging individual routing behavior with system-level efficiency through learning-based sequential assignment.