# State Aggregation for Distributed Value Iteration in Dynamic Programming

arXiv: https://arxiv.org/abs/2303.10675 · HTML: https://mlsanity.com/paper/2303.10675/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2023.

## Built on by (most cited first, 1 of 1)

- [SPoRt -- Safe Policy Ratio: Certified Training and Deployment of Task Policies in Model-Free RL](https://mlsanity.com/paper/2504.06386/) (2025)

## Abstract (arXiv)

We propose a distributed algorithm to solve a dynamic programming problem with multiple agents, where each agent has only partial knowledge of the state transition probabilities and costs. We provide consensus proofs for the presented algorithm and derive error bounds of the obtained value function with respect to what is considered as the "true solution" obtained from conventional value iteration. To minimize communication overhead between agents, state costs are aggregated and shared between agents only when the updated costs are expected to influence the solution of other agents significantly. We demonstrate the efficacy of the proposed distributed aggregation method to a large-scale urban traffic routing problem. Individual agents compute the fastest route to a common access point and share local congestion information with other agents allowing for fully distributed routing with minimal communication between agents.