mlsanityData to 7 Oct 2026

Efficient Duple Perturbation Robustness in Low-rank MDPs

arXiv 2404.08089 ↗First posted 11 Apr 2024cs.LG math.OCcited by 0

Not on the map. Why?

Abstract (arXiv)

The pursuit of robustness has recently been a popular topic in reinforcement learning (RL) research, yet the existing methods generally suffer from efficiency issues that obstruct their real-world implementation. In this paper, we introduce duple perturbation robustness, i.e. perturbation on both the feature and factor vectors for low-rank Markov decision processes (MDPs), via a novel characterization of (xi,η)-ambiguity sets. The novel robust MDP formulation is compatible with the function representation view, and therefore, is naturally applicable to practical RL problems with large or even continuous state-action spaces. Meanwhile, it also gives rise to a provably efficient and practical algorithm with theoretical convergence rate guarantee. Examples are designed to justify the new robustness concept, and algorithmic efficiency is supported by both theoretical bounds and numerical simulations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 50 references that are in the map. First posted on arXiv 11 Apr 2024.

Builds on

25 of 50 references.

References · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  3. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  4. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  5. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
  6. 2017Robust Adversarial Reinforcement Learningcited by 439
  7. 2019Online Learning: A Modern Introduction Using Convex Optimizationcited by 407
  8. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  9. 2020FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPscited by 232
  10. 2019Reinforcement Learning in Feature Space: Matrix Bandit, Kernels, and Regret Boundcited by 199
  11. 2020Fast Global Convergence of Natural Policy Gradient Methods with Entropy Regularizationcited by 170
  12. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 152
13 more
  1. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 136
  2. 2021Representation Learning for Online and Offline RL in Low-rank MDPscited by 128
  3. 2020Optimistic Policy Optimization with Bandit Feedbackcited by 79
  4. 2021Accelerating Robotic Reinforcement Learning via Parameterized Action Primitivescited by 63
  5. 2023Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coveragecited by 51
  6. 2020Partial Policy Iteration for L1-Robust Markov Decision Processescited by 49
  7. 2021Leveraging Non-uniformity in First-order Non-convex Optimizationcited by 44
  8. 2022Spectral Decomposition Representation for Reinforcement Learningcited by 35
  9. 2022Distributionally Robust Offline Reinforcement Learning with Linear Function Approximationcited by 31
  10. 2021A Free Lunch from the Noise: Provable and Practical Exploration for Representation Learningcited by 24
  11. 2022Latent Variable Representation for Reinforcement Learningcited by 14
  12. 2023Optimistic Natural Policy Gradient: a Simple Efficient Policy Optimization Framework for Online RLcited by 13
  13. 2019An Optimality Gap Test for a Semidefinite Relaxation of a Quadratic Program with Two Quadratic Constraintscited by 1

Built on by

Not cited yet.