mlsanityData to 7 Oct 2026

Distributionally Robust Reinforcement Learning

arXiv 1902.08708 ↗First posted 23 Feb 2019stat.ML cs.LGcited by 40

Not on the map. Why?

Abstract (arXiv)

Real-world applications require RL algorithms to act safely. During learning process, it is likely that the agent executes sub-optimal actions that may lead to unsafe/poor states of the system. Exploration is particularly brittle in high-dimensional state/action space due to increased number of low-performing actions. In this work, we consider risk-averse exploration in approximate RL setting. To ensure safety during learning, we propose the distributionally robust policy iteration scheme that provides lower bound guarantee on state-values. Our approach induces a dynamic level of risk to prevent poor decisions and yet preserves the convergence to the optimal policy. Our formulation results in a efficient algorithm that accounts for a simple re-weighting of policy actions in the standard policy iteration scheme. We extend our approach to continuous state/action space and present a practical algorithm, distributionally robust soft actor-critic, that implements a different exploration strategy: it acts conservatively at short-term and it explores optimistically in a long-run. We provide promising experimental results on continuous control tasks.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 8; 2023: 5; 2024: 4; 2025: 12; 2026: 5.

40 papers.

Sort:

By area:

  1. 2022Distributionally Robust Model-Based Offline Reinforcement Learning with Near-Optimal Sample Complexitycited by 85
  2. 2023The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Modelcited by 50
  3. 2021Sinkhorn Distributionally Robust Optimizationcited by 38
  4. 2022A Unified Wasserstein Distributional Robustness Framework for Adversarial Trainingcited by 36
  5. 2022Doubly Robust Distributionally Robust Off-Policy Evaluation and Learningcited by 34
  6. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  7. 2021Towards Theoretical Understandings of Robust Markov Decision Processes: Sample Complexity and Asymptoticscited by 27
  8. 2023Seeing is not Believing: Robust Reinforcement Learning against Spurious Correlationcited by 25
  9. 2024Sample-Efficient Robust Multi-Agent Reinforcement Learning in the Face of Environmental Uncertaintycited by 24
  10. 2022Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Beliefcited by 22
  11. 2021Robust Risk-Aware Reinforcement Learningcited by 17
  12. 2023Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackerscited by 15
28 more
  1. 2021No-Regret Reinforcement Learning with Heavy-Tailed Rewardscited by 13
  2. 2024Sample Complexity of Offline Distributionally Robust Linear Markov Decision Processescited by 12
  3. 2025Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learningcited by 12
  4. 2022Robust Reinforcement Learning with Distributional Risk-averse formulationcited by 9
  5. 2025SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfercited by 8
  6. 2022Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variablescited by 6
  7. 2023Extreme Risk Mitigation in Reinforcement Learning using Extreme Value Theorycited by 5
  8. 2025Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approachcited by 5
  9. 2025Distributionally Robust Optimization via Diffusion Ambiguity Modelingcited by 4
  10. 2023Pitfall of Optimism: Distributional Reinforcement Learning by Randomizing Risk Criterioncited by 3
  11. 2025Causality-Inspired Robustness for Nonlinear Models via Representation Learningcited by 3
  12. 2025DR-SAC: Distributionally Robust Soft Actor-Critic for Reinforcement Learning under Uncertaintycited by 3
  13. 2026Robust Policy Optimization to Prevent Catastrophic Forgettingcited by 3
  14. 2026Distributionally Robust Optimization via Generative Ambiguity Modelingcited by 3
  15. 2021Robust Reinforcement Learning under model misspecificationcited by 2
  16. 2025Distributionally Robust Multi-Agent Reinforcement Learning for Dynamic Chute Mappingcited by 2
  17. 2026DiagEval: Trajectory-Conditioned Diagnosis for Reliable Software Evaluation with GUI Agentscited by 2
  18. 2021Learning Provably Robust Motion Planners Using Funnel Librariescited by 1
  19. 2024Robust Reinforcement Learning with Dynamic Distortion Risk Measurescited by 1
  20. 2025Distributionally Robust Deep Q-Learningcited by 1
  21. 2025Distributionally Robust Self Paced Curriculum Reinforcement Learningcited by 1
  22. 2022On Practical Reinforcement Learning: Provable Robustness, Scalability, and Statistical Efficiencycited by 0
  23. 2024Survival of the Fittest: Evolutionary Adaptation of Policies for Environmental Shiftscited by 0
  24. 2025Distributionally Robust Federated Learning: An ADMM Algorithmcited by 0
  25. 2025Contraction-Aware Reinforcement Learning for Nonlinear Control with Statistical Robustnesscited by 0
  26. 2025Robust Reinforcement Learning in Finance: Modeling Market Impact with Elliptic Uncertainty Setscited by 0
  27. 2026Distributionally Robust Cooperative Multi-Agent Reinforcement Learning via Robust Value Factorizationcited by 0
  28. 2026Fuz-RL: A Fuzzy-Guided Robust Framework for Safe Reinforcement Learning under Uncertaintycited by 0