mlsanityData to 7 Oct 2026

Wasserstein Robust Reinforcement Learning

arXiv 1907.13196 ↗First posted 30 Jul 2019cs.LG cs.AI stat.MLcited by 50

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning algorithms, though successful, tend to over-fit to training environments hampering their application to the real-world. This paper proposes WR2L -- a robust reinforcement learning algorithm with significant robust performance on low and high-dimensional control tasks. Our method formalises robust reinforcement learning as a novel min-max game with a Wasserstein constraint for a correct and convergent solver. Apart from the formulation, we also propose an efficient and scalable solver following a novel zero-order optimisation method that we believe can be useful to numerical optimisation in general. We empirically demonstrate significant gains compared to standard and robust state-of-the-art algorithms on high-dimensional MuJuCo environments.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 11; 2023: 10; 2024: 9; 2025: 6; 2026: 7.

50 papers.

Sort:

  1. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  2. 2021Online Robust Reinforcement Learning with Model Uncertaintycited by 95
  3. 2022Policy Gradient Method For Robust Reinforcement Learningcited by 67
  4. 2021Sinkhorn Distributionally Robust Optimizationcited by 38
  5. 2022A Unified Wasserstein Distributional Robustness Framework for Adversarial Trainingcited by 36
  6. 2023Natural Actor-Critic for Robust Reinforcement Learning with Function Approximationcited by 35
  7. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  8. 2023Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithmscited by 25
  9. 2023Robust Average-Reward Markov Decision Processescited by 22
  10. 2021On the Complexity of Computing Markov Perfect Equilibrium in General-Sum Stochastic Gamescited by 18
  11. 2021Learning Robust Policy against Disturbance in Transition Dynamics via State-Conservative Policy Optimizationcited by 18
  12. 2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 18
38 more
  1. 2021Robust Risk-Aware Reinforcement Learningcited by 17
  2. 2022Risk-Sensitive Reinforcement Learning with Exponential Criteriacited by 15
  3. 2023Single-Trajectory Distributionally Robust Reinforcement Learningcited by 15
  4. 2022Robust Constrained Reinforcement Learningcited by 14
  5. 2022Robust Reinforcement Learning with Distributional Risk-averse formulationcited by 9
  6. 2023Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guaranteescited by 9
  7. 2024Robust Deep Reinforcement Learning with Adaptive Adversarial Perturbations in Action Spacecited by 9
  8. 2022Sparse Adversarial Attack in Multi-agent Reinforcement Learningcited by 8
  9. 2022Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variablescited by 6
  10. 2025ORVIT: Near-Optimal Online Distributionally Robust Reinforcement Learningcited by 5
  11. 2023DiAReL: Reinforcement Learning with Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Controlcited by 4
  12. 2024Communication-Efficient Byzantine-Resilient Federated Zero-Order Optimizationcited by 4
  13. 2025Wasserstein Adaptive Value Estimation for Actor-Critic Reinforcement Learningcited by 4
  14. 2022Truly Deterministic Policy Optimizationcited by 3
  15. 2023A Conservative Approach for Few-Shot Transfer in Off-Dynamics Reinforcement Learningcited by 3
  16. 2024Time-Constrained Robust MDPscited by 2
  17. 2026Wasserstein Proximal Policy Gradientcited by 2
  18. 2026Robust Adversarial Policy Optimization Under Dynamics Uncertaintycited by 2
  19. 2024How does Your RL Agent Explore? An Optimal Transport Analysis of Occupancy Measure Trajectoriescited by 1
  20. 2024Bootstrapping Expectiles in Reinforcement Learningcited by 1
  21. 2024Tolerance of Reinforcement Learning Controllers against Deviations in Cyber Physical Systemscited by 1
  22. 2024Robust Reinforcement Learning with Dynamic Distortion Risk Measurescited by 1
  23. 2025Risk-Averse Constrained Reinforcement Learning with Optimized Certainty Equivalentscited by 1
  24. 2025Distributionally Robust Self Paced Curriculum Reinforcement Learningcited by 1
  25. 2025Online Robust Reinforcement Learning with General Function Approximationcited by 1
  26. 2021L2E: Learning to Exploit Your Opponentcited by 0
  27. 2022User-Oriented Robust Reinforcement Learningcited by 0
  28. 2022Bounded Robustness in Reinforcement Learning via Lexicographic Objectivescited by 0
  29. 2023Risk-Averse MDPs under Reward Ambiguitycited by 0
  30. 2023On Practical Robust Reinforcement Learning: Practical Uncertainty Set and Double-Agent Algorithmcited by 0
  31. 2024Lipschitz-Regularized Critics Lead to Policy Robustness Against Transition Dynamics Uncertaintycited by 0
  32. 2024Zero-Sum Positional Differential Games as a Framework for Robust Reinforcement Learning: Deep Q-Learning Approachcited by 0
  33. 2025Robust Reinforcement Learning in Finance: Modeling Market Impact with Elliptic Uncertainty Setscited by 0
  34. 2026Reward-Preserving Attacks For Robust Reinforcement Learningcited by 0
  35. 2026Distributional Off-Policy Evaluation with Deep Quantile Process Regressioncited by 0
  36. 2026Beyond Coefficients: Forecast-Necessity Testing for Interpretable Causal Discovery in Nonlinear Time-Series Modelscited by 0
  37. 2026Stationary Robust Mean-Field Games under Model Mismatchescited by 0
  38. 2026Robust Risk Under Evolving Uncertainty: A Wasserstein Counterpart of the Entropic Value-at-Riskcited by 0