mlsanityData to 13 Oct 2026

Refined Policy Improvement Bounds for MDPs

arXiv 2107.08068 ↗First posted 16 Jul 2021cs.LG cs.AI math.OCcited by 3

Not on the map. Why?

Abstract (arXiv)

The policy improvement bound on the difference of the discounted returns plays a crucial role in the theoretical justification of the trust-region policy optimization (TRPO) algorithm. The existing bound leads to a degenerate bound when the discount factor approaches one, making the applicability of TRPO and related algorithms questionable when the discount factor is close to one. We refine the results in citeSchulman2015, Achiam2017 and propose a novel bound that is "continuous" in the discount factor. In particular, our bound is applicable for MDPs with the long-run average rewards as well.

Key facts. Cited by 3 papers in the map (data through 13 Oct 2026). Builds on 7 of its 17 references that are in the map. First posted on arXiv 16 Jul 2021.

Builds on

7 of 17 references.

References · 7

  1. 2017Proximal Policy Optimization Algorithmscited by 14,592
  2. 2015Trust Region Policy Optimizationcited by 2,513
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,940
  4. 2017Constrained Policy Optimizationcited by 931
  5. 2018Maximum a Posteriori Policy Optimisationcited by 320
  6. 2020Queueing Network Controls via Deep Reinforcement Learningcited by 40
  7. 2021On-Policy Deep Reinforcement Learning for the Average-Reward Criterioncited by 29

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 0; 2025: 1.

3 papers.

  1. 2023Policy Optimization for Continuous Reinforcement Learningcited by 31
  2. 2025Enhancing PPO with Trajectory-Aware Hybrid Policiescited by 4
  3. 2022Processing Network Controls via Deep Reinforcement Learningcited by 0