# Multiagent Reinforcement Learning for Autonomous Routing and Pickup Problem with Adaptation to Variable Demand

arXiv: https://arxiv.org/abs/2211.14983 · HTML: https://mlsanity.com/paper/2211.14983/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 44 references that are in the map.
- First posted on arXiv 28 Nov 2022.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations](https://mlsanity.com/paper/1505.05116/) (2015)
- [DESPOT: Online POMDP Planning with Regularization](https://mlsanity.com/paper/1609.03250/) (2016)
- [Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control](https://mlsanity.com/paper/2108.10315/) (2021)
- [Efficient Large-Scale Multi-Drone Delivery Using Transit Networks](https://mlsanity.com/paper/1909.11840/) (2019)
- [Multiagent Rollout and Policy Iteration for POMDP with Application to Multi-Robot Repair Problems](https://mlsanity.com/paper/2011.04222/) (2020)
- [Multiagent Value Iteration Algorithms in Dynamic Programming and Reinforcement Learning](https://mlsanity.com/paper/2005.01627/) (2020)
- [Multiagent Rollout Algorithms and Reinforcement Learning](https://mlsanity.com/paper/1910.00120/) (2019)
- [Reinforcement Learning for POMDP: Partitioned Rollout and Policy Iteration with Application to Autonomous Sequential Repair Problems](https://mlsanity.com/paper/2002.04175/) (2020)
- [Constrained Multiagent Rollout and Multidimensional Assignment with the Auction Algorithm](https://mlsanity.com/paper/2002.07407/) (2020)
- [Multiagent Rollout with Reshuffling for Warehouse Robots Path Planning](https://mlsanity.com/paper/2211.08201/) (2022)

## Built on by (most cited first, 6 of 6)

- [Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities](https://mlsanity.com/paper/2506.18019/) (2025)
- [Model Predictive Control and Reinforcement Learning: A Unified Framework Based on Dynamic Programming](https://mlsanity.com/paper/2406.00592/) (2024)
- [Data-Efficient Multi-Agent Spatial Planning with LLMs](https://mlsanity.com/paper/2502.18822/) (2025)
- [Surge Routing: Event-informed Multiagent Reinforcement Learning for Autonomous Rideshare](https://mlsanity.com/paper/2307.02637/) (2023)
- [Approximate Multiagent Reinforcement Learning for On-Demand Urban Mobility Problem on a Large Map (extended version)](https://mlsanity.com/paper/2311.01534/) (2023)
- [Pro-Routing: Proactive Routing of Autonomous Multi-Capacity Robots for Pickup-and-Delivery Tasks](https://mlsanity.com/paper/2503.24325/) (2025)

## Abstract (arXiv)

We derive a learning framework to generate routing/pickup policies for a fleet of autonomous vehicles tasked with servicing stochastically appearing requests on a city map. We focus on policies that 1) give rise to coordination amongst the vehicles, thereby reducing wait times for servicing requests, 2) are non-myopic, and consider a-priori potential future requests, 3) can adapt to changes in the underlying demand distribution. Specifically, we are interested in policies that are adaptive to fluctuations of actual demand conditions in urban environments, such as on-peak vs. off-peak hours. We achieve this through a combination of (i) an online play algorithm that improves the performance of an offline-trained policy, and (ii) an offline approximation scheme that allows for adapting to changes in the underlying demand model. In particular, we achieve adaptivity of our learned policy to different demand distributions by quantifying a region of validity using the q-valid radius of a Wasserstein Ambiguity Set. We propose a mechanism for switching the originally trained offline approximation when the current demand is outside the original validity region. In this case, we propose to use an offline architecture, trained on a historical demand model that is closer to the current demand in terms of Wasserstein distance. We learn routing and pickup policies over real taxicab requests in San Francisco with high variability between on-peak and off-peak hours, demonstrating the ability of our method to adapt to real fluctuation in demand distributions. Our numerical results demonstrate that our method outperforms alternative rollout-based reinforcement learning schemes, as well as other classical methods from operations research.