mlsanityData to 7 Oct 2026

Policy Optimization with Stochastic Mirror Descent

arXiv 1906.10462 ↗First posted 25 Jun 2019cs.LG stat.MLcited by 17

Not on the map. Why?

Abstract (arXiv)

Improving sample efficiency has been a longstanding goal in reinforcement learning. This paper proposes mathttVRMPO algorithm: a sample efficient policy gradient method with stochastic mirror descent. In mathttVRMPO, a novel variance-reduced policy gradient estimator is presented to improve sample efficiency. We prove that the proposed mathttVRMPO needs only O(ε-3) sample trajectories to achieve an ε-approximate first-order stationary point, which matches the best sample complexity for policy optimization. The extensive experimental results demonstrate that mathttVRMPO outperforms the state-of-the-art policy gradient methods in various settings.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 5; 2023: 2; 2024: 4; 2025: 2; 2026: 2.

17 papers.

Sort:

  1. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  2. 2021A general sample complexity analysis of vanilla policy gradientcited by 74
  3. 2022Constrained Update Projection Approach to Safe Policy Optimizationcited by 55
  4. 2023A Novel Framework for Policy Mirror Descent with General Parameterization and Linear Convergencecited by 29
  5. 2021Bregman Gradient Policy Optimizationcited by 15
  6. 2022On the Hidden Biases of Policy Mirror Ascent in Continuous Action Spacescited by 11
  7. 2022Momentum-Based Policy Gradient with Second-Order Informationcited by 11
  8. 2022Learning to Constrain Policy Optimization with Virtual Trust Regioncited by 4
  9. 2024FlagVNE: A Flexible and Generalizable Reinforcement Learning Framework for Network Resource Allocationcited by 2
  10. 2025Policy Mirror Descent with Temporal Difference Learning: Sample Complexity under Online Markov Datacited by 2
  11. 2023Efficiently Escaping Saddle Points for Policy Optimizationcited by 1
  12. 2024On the Stochastic (Variance-Reduced) Proximal Gradient Method for Regularized Expected Reward Optimizationcited by 1
5 more
  1. 2026Safe, Scalable, and Accurate Bayes Posterior Sampling for Large-Data Generalized Linear Mixed Modelscited by 1
  2. 2024Off-OAB: Off-Policy Policy Gradient Method with Optimal Action-Dependent Baselinecited by 0
  3. 2024Heavy-Ball Momentum Accelerated Actor-Critic With Function Approximationcited by 0
  4. 2025Mirror Descent Actor Critic via Bounded Advantage Learningcited by 0
  5. 2026Mirror Descent on Riemannian Manifoldscited by 0