# The Value Iteration Algorithm is Not Strongly Polynomial for Discounted Dynamic Programming

arXiv: https://arxiv.org/abs/1312.6832 · HTML: https://mlsanity.com/paper/1312.6832/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Dec 2013.

## Built on by (most cited first, 4 of 4)

- [On the Hardness of Reinforcement Learning with Transition Look-Ahead](https://mlsanity.com/paper/2510.19372/) (2025)
- [MDP Geometry, Normalization and Reward Balancing Solvers](https://mlsanity.com/paper/2407.06712/) (2024)
- [Geometric Re-Analysis of Classical MDP Solving Algorithms](https://mlsanity.com/paper/2503.04203/) (2025)
- [Revisiting Value Iteration: Unified Analysis of Discounted and Average-Reward Cases](https://mlsanity.com/paper/2510.23914/) (2025)

## Abstract (arXiv)

This note provides a simple example demonstrating that, if exact computations are allowed, the number of iterations required for the value iteration algorithm to find an optimal policy for discounted dynamic programming problems may grow arbitrarily quickly with the size of the problem. In particular, the number of iterations can be exponential in the number of actions. Thus, unlike policy iterations, the value iteration algorithm is not strongly polynomial for discounted dynamic programming.