# Autonomous vehicles need social awareness to find optima in multi-agent reinforcement learning routing games

arXiv: https://arxiv.org/abs/2510.11410 · HTML: https://mlsanity.com/paper/2510.11410/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 5 of its 41 references that are in the map.
- First posted on arXiv 13 Oct 2025.

## Placed in

- End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/)

## Builds on (most cited first, 5 of 5)

- [Learning How to Dynamically Route Autonomous Vehicles on Shared Roads](https://mlsanity.com/paper/1909.03664/) (2019)
- [URB -- Urban Routing Benchmark for RL-equipped Connected Autonomous Vehicles](https://mlsanity.com/paper/2505.17734/) (2025)
- [RouteRL: Multi-agent reinforcement learning framework for urban route choice with autonomous vehicles](https://mlsanity.com/paper/2502.20065/) (2025)
- [Collaboration Between the City and Machine Learning Community is Crucial to Efficient Autonomous Vehicles Routing](https://mlsanity.com/paper/2502.13188/) (2025)
- [Wardropian Cycles make traffic assignment both optimal and fair by eliminating price-of-anarchy with Cyclical User Equilibrium for compliant connected autonomous vehicles](https://mlsanity.com/paper/2507.19675/) (2025)

## Built on by (most cited first, 1 of 1)

- [Collective Autonomous Vehicle Routing Needs Social Awareness for Coexistence with Human Drivers](https://mlsanity.com/paper/2610.13439/) (2026)

## Abstract (arXiv)

Previous work has shown that when multiple selfish Autonomous Vehicles (AVs) are introduced to future cities and start learning optimal routing strategies using Multi-Agent Reinforcement Learning (MARL), they may destabilize traffic systems, as they would require a significant amount of time to converge to the optimal solution, equivalent to years of real-world commuting. We demonstrate that moving beyond the selfish component in the reward significantly relieves this issue. If each AV, apart from minimizing its own travel time, aims to reduce its impact on the system, this will be beneficial not only for the system-wide performance but also for each individual player in this routing game. By introducing an intrinsic reward signal based on the marginal cost matrix, we significantly reduce training time and achieve convergence more reliably. Marginal cost quantifies the impact of each individual action (route-choice) on the system (total travel time). Including it as one of the components of the reward can reduce the degree of non-stationarity by aligning agents' objectives. Notably, the proposed counterfactual formulation preserves the system's equilibria and avoids oscillations. Our experiments show that training MARL algorithms with our novel reward formulation enables the agents to converge to the optimal solution, whereas the baseline algorithms fail to do so. We show these effects in both a toy network and the real-world network of Saint-Arnoult. Our results optimistically indicate that social awareness (i.e., including marginal costs in routing decisions) improves both the system-wide and individual performance of future urban systems with AVs.