# Hierarchical Planning for Dynamic Resource Allocation in Smart and Connected Communities

arXiv: https://arxiv.org/abs/2107.01292 · HTML: https://mlsanity.com/paper/2107.01292/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2021.

## Built on by (most cited first, 3 of 3)

- [Decision Making in Non-Stationary Environments with Policy-Augmented Search](https://mlsanity.com/paper/2401.03197/) (2024)
- [Multi-Agent Reinforcement Learning with Hierarchical Coordination for Emergency Responder Stationing](https://mlsanity.com/paper/2405.13205/) (2024)
- [Decision Making in Non-Stationary Environments with Policy-Augmented Monte Carlo Tree Search](https://mlsanity.com/paper/2202.13003/) (2022)

## Abstract (arXiv)

Resource allocation under uncertainty is a classical problem in city-scale cyber-physical systems. Consider emergency response as an example; urban planners and first responders optimize the location of ambulances to minimize expected response times to incidents such as road accidents. Typically, such problems deal with sequential decision-making under uncertainty and can be modeled as Markov (or semi-Markov) decision processes. The goal of the decision-maker is to learn a mapping from states to actions that can maximize expected rewards. While online, offline, and decentralized approaches have been proposed to tackle such problems, scalability remains a challenge for real-world use-cases. We present a general approach to hierarchical planning that leverages structure in city-level CPS problems for resource allocation. We use emergency response as a case study and show how a large resource allocation problem can be split into smaller problems. We then use Monte-Carlo planning for solving the smaller problems and managing the interaction between them. Finally, we use data from Nashville, Tennessee, a major metropolitan area in the United States, to validate our approach. Our experiments show that the proposed approach outperforms state-of-the-art approaches used in the field of emergency response.