# Finding the Near Optimal Policy via Adaptive Reduced Regularization in MDPs

arXiv: https://arxiv.org/abs/2011.00213 · HTML: https://mlsanity.com/paper/2011.00213/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2020.

## Built on by (most cited first, 4 of 4)

- [Fast Policy Extragradient Methods for Competitive Games with Entropy Regularization](https://mlsanity.com/paper/2105.15186/) (2021)
- [Softmax Policy Gradient Methods Can Take Exponential Time to Converge](https://mlsanity.com/paper/2102.11270/) (2021)
- [The Power of Regularization in Solving Extensive-Form Games](https://mlsanity.com/paper/2206.09495/) (2022)
- [Finite-Time Analysis of Entropy-Regularized Neural Natural Actor-Critic Algorithm](https://mlsanity.com/paper/2206.00833/) (2022)

## Abstract (arXiv)

Regularized MDPs serve as a smooth version of original MDPs. However, biased optimal policy always exists for regularized MDPs. Instead of making the coefficientλof regularized term sufficiently small, we propose an adaptive reduction scheme for λ to approximate optimal policy of the original MDP. It is shown that the iteration complexity for obtaining anε-optimal policy could be reduced in comparison with setting sufficiently smallλ. In addition, there exists strong duality connection between the reduction method and solving the original MDP directly, from which we can derive more adaptive reduction method for certain algorithms.