mlsanityData to 7 Oct 2026

Stochastic Recursive Momentum for Policy Gradient Methods

arXiv 2003.04302 ↗First posted 9 Mar 2020stat.ML cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a novel algorithm named STOchastic Recursive Momentum for Policy Gradient (STORM-PG), which operates a SARAH-type stochastic recursive variance-reduced policy gradient in an exponential moving average fashion. STORM-PG enjoys a provably sharp O(1/ε3) sample complexity bound for STORM-PG, matching the best-known convergence rate for policy gradient algorithm. In the mean time, STORM-PG avoids the alternations between large batches and small batches which persists in comparable variance-reduced policy gradient methods, allowing considerably simpler parameter tuning. Numerical experiments depicts the superiority of our algorithm over comparative policy gradient algorithms.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 5; 2023: 4; 2024: 5; 2025: 1; 2026: 3.

22 papers.

Sort:

  1. 2021A general sample complexity analysis of vanilla policy gradientcited by 74
  2. 2022An Improved Analysis of (Variance-Reduced) Policy Gradient and Natural Policy Gradient Methodscited by 63
  3. 2023Stochastic Policy Gradient Methods: Improved Sample Complexity for Fisher-non-degenerate Policiescited by 53
  4. 2023Improved Sample Complexity Analysis of Natural Policy Gradient Algorithm with General Parameterization for Infinite Horizon Discounted Reward Markov Decision Processescited by 24
  5. 2021On the Global Optimum Convergence of Momentum-based Policy Gradientcited by 23
  6. 2023Reinforcement Learning with General Utilities: Simpler Variance Reduction and Large State-Action Spacecited by 18
  7. 2022PAGE-PG: A Simple and Loopless Variance-Reduced Policy Gradient Method with Probabilistic Gradient Estimationcited by 15
  8. 2022On the Hidden Biases of Policy Mirror Ascent in Continuous Action Spacescited by 11
  9. 2022Momentum-Based Policy Gradient with Second-Order Informationcited by 11
  10. 2024Policy Gradient with Active Importance Samplingcited by 9
  11. 2024Momentum for the Win: Collaborative Federated Reinforcement Learning across Heterogeneous Environmentscited by 7
  12. 2021On the Convergence Rate of Off-Policy Policy Optimization Methods with Density-Ratio Correctioncited by 4
10 more
  1. 2024Global Convergence of Natural Policy Gradient with Hessian-aided Momentum Variance Reductioncited by 4
  2. 2022Dealing with Sparse Rewards in Continuous Control Robotics via Heavy-Tailed Policiescited by 2
  3. 2024Decentralized Federated Policy Gradient with Byzantine Fault-Tolerance and Provably Fast Convergencecited by 2
  4. 2024On the Stochastic (Variance-Reduced) Proximal Gradient Method for Regularized Expected Reward Optimizationcited by 1
  5. 2025Reusing Trajectories in Policy Gradients Enables Fast Convergencecited by 1
  6. 2021Adaptive Stochastic ADMM for Decentralized Reinforcement Learning in Edge Industrial IoTcited by 0
  7. 2023Stochastic Dimension-reduced Second-order Methods for Policy Optimizationcited by 0
  8. 2026Variance-Reduced Conditional Gradient Methods under Markovian Sampling for Nonconvex Composite Optimizationcited by 0
  9. 2026Learning Chance-Constrained MDPs with Bellman Distributional Certificatescited by 0
  10. 2026Reusing Past Samples in Proximal Policy Optimization: When and How Does It Help?cited by 0