mlsanityData to 14 Oct 2026

An Efficient Solution to s-Rectangular Robust Markov Decision Processes

arXiv 2301.13642 ↗First posted 31 Jan 2023cs.LG math.OCcited by 6

Not on the map. Why?

Abstract (arXiv)

We present an efficient robust value iteration for texttts-rectangular robust Markov Decision Processes (MDPs) with a time complexity comparable to standard (non-robust) MDPs which is significantly faster than any existing method. We do so by deriving the optimal robust Bellman operator in concrete forms using our Lp water filling lemma. We unveil the exact form of the optimal policies, which turn out to be novel threshold policies with the probability of playing an action proportional to its advantage.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 13 of its 31 references that are in the map. First posted on arXiv 31 Jan 2023.

Builds on

13 of 31 references.

References · 13

  1. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  2. 2017Minimax Regret Bounds for Reinforcement Learningcited by 503
  3. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176
  4. 2017Equivalence Between Policy Gradients and Soft Q-Learningcited by 173
  5. 2018Assessing Generalization in Deep Reinforcement Learningcited by 133
  6. 2021Online Robust Reinforcement Learning with Model Uncertaintycited by 96
  7. 2022Policy Gradient Method For Robust Reinforcement Learningcited by 69
  8. 2020Partial Policy Iteration for L1-Robust Markov Decision Processescited by 50
  9. 2021Twice regularized MDPs and the equivalence between robustness and regularizationcited by 47
  10. 2020Distributional Robustness and Regularization in Reinforcement Learningcited by 40
  11. 2018Robust Markov Decision Process: Beyond Rectangularitycited by 24
  12. 2019Model-Based Reinforcement Learning Exploiting State-Action Equivalencecited by 6
1 more
  1. 2021Robust Entropy-regularized Markov Decision Processescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025Model-Free Robust Average-Reward Reinforcement Learning with Sample Complexity Analysiscited by 3
  2. 2025Pessimism Principle Can Be Effective: Towards a Framework for Zero-Shot Transfer Reinforcement Learningcited by 2
  3. 2025Distributionally Robust Deep Q-Learningcited by 1
  4. 2025ADARL: Adaptive Low-Rank Structures for Robust Policy Learning under Uncertaintycited by 0
  5. 2025Robust Reinforcement Learning in Finance: Modeling Market Impact with Elliptic Uncertainty Setscited by 0
  6. 2026Vector Bellman Theory for Multichain Robust Average-Reward Markov Decision Processescited by 0