# RouteRL: Multi-agent reinforcement learning framework for urban route choice with autonomous vehicles

arXiv: https://arxiv.org/abs/2502.20065 · HTML: https://mlsanity.com/paper/2502.20065/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 13 of its 29 references that are in the map.
- First posted on arXiv 27 Feb 2025.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.08839/) (2020)
- [Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?](https://mlsanity.com/paper/2011.09533/) (2020)
- [PettingZoo: Gym for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2009.14471/) (2020)
- [Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks](https://mlsanity.com/paper/2006.07869/) (2020)
- [SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving](https://mlsanity.com/paper/2010.09776/) (2020)
- [Emergent Complexity via Multi-Agent Competition](https://mlsanity.com/paper/1710.03748/) (2017)
- [CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario](https://mlsanity.com/paper/1905.05217/) (2019)
- [TorchRL: A data-driven decision-making library for PyTorch](https://mlsanity.com/paper/2306.00577/) (2023)
- [Neural MMO v1.3: A Massively Multiagent Game Environment for Training and Evaluating Neural Networks](https://mlsanity.com/paper/2001.12004/) (2020)
- [Wardrop Equilibrium Can Be Boundedly Rational: A New Behavioral Theory of Route Choice](https://mlsanity.com/paper/2304.02500/) (2023)
- [Social impact of CAVs -- coexistence of machines and humans in the context of route choice](https://mlsanity.com/paper/2409.12839/) (2024)

## Built on by (most cited first, 3 of 3)

- [URB -- Urban Routing Benchmark for RL-equipped Connected Autonomous Vehicles](https://mlsanity.com/paper/2505.17734/) (2025)
- [Autonomous vehicles need social awareness to find optima in multi-agent reinforcement learning routing games](https://mlsanity.com/paper/2510.11410/) (2025)
- [Collective Autonomous Vehicle Routing Needs Social Awareness for Coexistence with Human Drivers](https://mlsanity.com/paper/2610.13439/) (2026)

## Abstract (arXiv)

RouteRL is a novel framework that integrates multi-agent reinforcement learning (MARL) with a microscopic traffic simulation, facilitating the testing and development of efficient route choice strategies for autonomous vehicles (AVs). The proposed framework simulates the daily route choices of driver agents in a city, including two types: human drivers, emulated using behavioral route choice models, and AVs, modeled as MARL agents optimizing their policies for a predefined objective. RouteRL aims to advance research in MARL, transport modeling, and human-AI interaction for transportation applications. This study presents a technical report on RouteRL, outlines its potential research contributions, and showcases its impact via illustrative examples.