# On Value Iteration Convergence in Connected MDPs

arXiv: https://arxiv.org/abs/2406.09592 · HTML: https://mlsanity.com/paper/2406.09592/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 3 references that are in the map.
- First posted on arXiv 13 Jun 2024.

## Built on by (most cited first, 1 of 1)

- [Beyond the Bellman Fixed Point: Geometry and Fast Policy Identification in Value Iteration](https://mlsanity.com/paper/2604.17457/) (2026)

## Abstract (arXiv)

This paper establishes that an MDP with a unique optimal policy and ergodic associated transition matrix ensures the convergence of various versions of the Value Iteration algorithm at a geometric rate that exceeds the discount factor γ for both discounted and average-reward criteria.