mlsanityData to 14 Oct 2026

Efficient Wasserstein Natural Gradients for Reinforcement Learning

arXiv 2010.05380 ↗First posted 12 Oct 2020cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

A novel optimization approach is proposed for application to policy gradient methods and evolution strategies for reinforcement learning (RL). The procedure uses a computationally efficient Wasserstein natural gradient (WNG) descent that takes advantage of the geometry induced by a Wasserstein penalty to speed optimization. This method follows the recent theme in RL of including a divergence penalty in the objective to establish a trust region. Experiments on challenging tasks demonstrate improvements in both computational cost and performance over advanced baselines.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 5; 2023: 8; 2024: 0; 2025: 2; 2026: 4.

23 papers.

Sort:

By area:

  1. 2023Confronting Reward Model Overoptimization with Constrained RLHFcited by 96
  2. 2021Tactical Optimism and Pessimism for Deep Reinforcement Learningcited by 55
  3. 2021MICo: Improved representations via sampling-based state similarity for Markov decision processescited by 43
  4. 2023ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPscited by 20
  5. 2023The Geometry of Neural Nets' Parameter Spaces Under Reparametrizationcited by 16
  6. 2021A First-Occupancy Representation for Reinforcement Learningcited by 14
  7. 2025Wasserstein Policy Optimizationcited by 13
  8. 2022Learning General World Models in a Handful of Reward-Free Deploymentscited by 12
  9. 2022Geometry and convergence of natural policy gradient methodscited by 12
  10. 2021Towards an Understanding of Default Policies in Multitask Policy Optimizationcited by 8
  11. 2022Trust Region Policy Optimization with Optimal Transport Discrepancies: Duality and Algorithm for Continuous Actionscited by 8
  12. 2023Wasserstein Gradient Flows for Optimizing Gaussian Mixture Policiescited by 8
11 more
  1. 2023Provably Convergent Policy Optimization via Metric-aware Trust Region Methodscited by 6
  2. 2026Wasserstein Proximal Policy Gradientcited by 4
  3. 2023A State Representation for Diminishing Rewardscited by 3
  4. 2023A Conservative Approach for Few-Shot Transfer in Off-Dynamics Reinforcement Learningcited by 3
  5. 2025Constrained Sliced Wasserstein Embeddingcited by 2
  6. 2026Semantic-aware Wasserstein Policy Regularization for Large Language Model Alignmentcited by 2
  7. 2022Minimum Description Length Controlcited by 0
  8. 2022Transfer RL via the Undo Maps Formalismcited by 0
  9. 2023Wasserstein Diversity-Enriched Regularizer for Hierarchical Reinforcement Learningcited by 0
  10. 2026Wasserstein Policy Gradient for Entropy-Regularized Linear-Quadratic Controlcited by 0
  11. 2026Multi-step Proximal Policy Improvement in Offline Reinforcement Learningcited by 0