mlsanityData to 14 Oct 2026

Bring Your Own (Non-Robust) Algorithm to Solve Robust MDPs by Estimating The Worst Kernel

arXiv 2306.05859 ↗First posted 9 Jun 2023cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Robust Markov Decision Processes (RMDPs) provide a framework for sequential decision-making that is robust to perturbations on the transition kernel. However, current RMDP methods are often limited to small-scale problems, hindering their use in high-dimensional domains. To bridge this gap, we present EWoK, a novel online approach to solve RMDP that Estimates the Worst transition Kernel to learn robust policies. Unlike previous works that regularize the policy or value updates, EWoK achieves robustness by simulating the worst scenarios for the agent while retaining complete flexibility in the learning process. Notably, EWoK can be applied on top of any off-the-shelf em non-robust RL algorithm, enabling easy scaling to high-dimensional domains. Our experiments, spanning from simple Cartpole to high-dimensional DeepMind Control Suite environments, demonstrate the effectiveness and applicability of the EWoK paradigm as a practical method for learning robust policies.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 30 of its 47 references that are in the map. First posted on arXiv 9 Jun 2023.

Builds on

30 of 47 references.

References · 30

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  5. 2016OpenAI Gymcited by 1,736
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  7. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  8. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  9. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  10. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  11. 2017Robust Adversarial Reinforcement Learningcited by 451
  12. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
18 more
  1. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  2. 2022RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learningcited by 115
  3. 2021Replay-Guided Adversarial Environment Designcited by 111
  4. 2021Online Robust Reinforcement Learning with Model Uncertaintycited by 96
  5. 2018Lipschitz Continuity in Model-based Reinforcement Learningcited by 91
  6. 2022Distributionally Robust Model-Based Offline Reinforcement Learning with Near-Optimal Sample Complexitycited by 87
  7. 2021Sample Complexity of Robust Reinforcement Learning with a Generative Modelcited by 83
  8. 2020An empirical investigation of the challenges of real-world reinforcement learningcited by 77
  9. 2022Policy Gradient Method For Robust Reinforcement Learningcited by 69
  10. 2019Risk Averse Robust Adversarial Reinforcement Learningcited by 57
  11. 2020Partial Policy Iteration for L1-Robust Markov Decision Processescited by 50
  12. 2021Twice regularized MDPs and the equivalence between robustness and regularizationcited by 47
  13. 2023Improved Sample Complexity Bounds for Distributionally Robust Reinforcement Learningcited by 44
  14. 2022Efficient Risk-Averse Reinforcement Learningcited by 43
  15. 2022Policy Gradient in Robust MDPs with Global Convergence Guaranteecited by 38
  16. 2020Scalable First-Order Methods for Robust MDPscited by 25
  17. 2022Efficient Policy Iteration for Robust Markov Decision Processes via Regularizationcited by 20
  18. 2022Robust Constrained Reinforcement Learningcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 3.

8 papers.

  1. 2025Robust LLM Alignment via Distributionally Robust Direct Preference Optimizationcited by 18
  2. 2025SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfercited by 8
  3. 2024A Single-Loop Robust Policy Gradient Method for Robust Markov Decision Processescited by 4
  4. 2024Robust Model-Based Reinforcement Learning with an Adversarial Auxiliary Modelcited by 3
  5. 2026Finite-State Controllers for (Hidden-Model) POMDPs using Deep Reinforcement Learningcited by 1
  6. 2025ADARL: Adaptive Low-Rank Structures for Robust Policy Learning under Uncertaintycited by 0
  7. 2026Post-Hoc Robustness for Model-Based Reinforcement Learningcited by 0
  8. 2026Vector Bellman Theory for Multichain Robust Average-Reward Markov Decision Processescited by 0