# Multi-Agent Reinforcement Learning with Hierarchical Coordination for Emergency Responder Stationing

arXiv: https://arxiv.org/abs/2405.13205 · HTML: https://mlsanity.com/paper/2405.13205/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 49 references that are in the map.
- First posted on arXiv 21 May 2024.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Stabilizing Transformers for Reinforcement Learning](https://mlsanity.com/paper/1910.06764/) (2019)
- [Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents](https://mlsanity.com/paper/1802.08757/) (2018)
- [A Review of Cooperative Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1908.03963/) (2019)
- [Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1901.11454/) (2019)
- [Efficient Transformers in Reinforcement Learning using Actor-Learner Distillation](https://mlsanity.com/paper/2104.01655/) (2021)
- [Multi-Agent Reinforcement Learning for Order-dispatching via Order-Vehicle Distribution Matching](https://mlsanity.com/paper/1910.02591/) (2019)
- [CoRide: Joint Order Dispatching and Fleet Management for Multi-Scale Ride-Hailing Platforms](https://mlsanity.com/paper/1905.11353/) (2019)
- [HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanism](https://mlsanity.com/paper/2110.07246/) (2021)
- [On Algorithmic Decision Procedures in Emergency Response Systems in Smart and Connected Communities](https://mlsanity.com/paper/2001.07362/) (2020)
- [Learning Incident Prediction Models Over Large Geographical Areas for Emergency Response Systems](https://mlsanity.com/paper/2106.08307/) (2021)
- [Hierarchical Planning for Dynamic Resource Allocation in Smart and Connected Communities](https://mlsanity.com/paper/2107.01292/) (2021)
- [Multi-Agent Reinforcement Learning for Assessing False-Data Injection Attacks on Transportation Networks](https://mlsanity.com/paper/2312.14625/) (2023)
- [An Online Approach to Solving Public Transit Stationing and Dispatch Problem](https://mlsanity.com/paper/2403.03339/) (2024)

## Built on by (most cited first, 4 of 4)

- [HiMA-Ecom: Enabling Joint Training of Hierarchical Multi-Agent E-commerce Assistants](https://mlsanity.com/paper/2506.19846/) (2025)
- [One Step is Enough: Multi-Agent Reinforcement Learning based on One-Step Policy Optimization for Order Dispatch on Ride-Sharing Platforms](https://mlsanity.com/paper/2507.15351/) (2025)
- [DispatchMAS: Fusing taxonomy and artificial intelligence agents for emergency medical services](https://mlsanity.com/paper/2510.21228/) (2025)
- [Hierarchical Adversarially-Resilient Multi-Agent Reinforcement Learning for Cyber-Physical Systems Security](https://mlsanity.com/paper/2506.22445/) (2025)

## Abstract (arXiv)

An emergency responder management (ERM) system dispatches responders, such as ambulances, when it receives requests for medical aid. ERM systems can also proactively reposition responders between predesignated waiting locations to cover any gaps that arise due to the prior dispatch of responders or significant changes in the distribution of anticipated requests. Optimal repositioning is computationally challenging due to the exponential number of ways to allocate responders between locations and the uncertainty in future requests. The state-of-the-art approach in proactive repositioning is a hierarchical approach based on spatial decomposition and online Monte Carlo tree search, which may require minutes of computation for each decision in a domain where seconds can save lives. We address the issue of long decision times by introducing a novel reinforcement learning (RL) approach, based on the same hierarchical decomposition, but replacing online search with learning. To address the computational challenges posed by large, variable-dimensional, and discrete state and action spaces, we propose: (1) actor-critic based agents that incorporate transformers to handle variable-dimensional states and actions, (2) projections to fixed-dimensional observations to handle complex states, and (3) combinatorial techniques to map continuous actions to discrete allocations. We evaluate our approach using real-world data from two U.S. cities, Nashville, TN and Seattle, WA. Our experiments show that compared to the state of the art, our approach reduces computation time per decision by three orders of magnitude, while also slightly reducing average ambulance response time by 5 seconds.