mlsanityData to 7 Oct 2026

Robust Policy Learning over Multiple Uncertainty Sets

arXiv 2202.07013 ↗First posted 14 Feb 2022cs.LG cs.AI cs.ROcited by 16

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) agents need to be robust to variations in safety-critical environments. While system identification methods provide a way to infer the variation from online experience, they can fail in settings where fast identification is not possible. Another dominant approach is robust RL which produces a policy that can handle worst-case scenarios, but these methods are generally designed to achieve robustness to a single uncertainty set that must be specified at train time. Towards a more general solution, we formulate the multi-set robustness problem to learn a policy robust to different perturbation sets. We then design an algorithm that enjoys the benefits of both system identification and robust RL: it reduces uncertainty where possible given a few interactions, but can still act robustly with respect to the remaining uncertainty. On a diverse set of control tasks, our approach demonstrates improved worst-case performance on new environments compared to prior methods based on system identification and on robust RL alone.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 44 of its 62 references that are in the map. First posted on arXiv 14 Feb 2022.

Builds on

44 of 62 references.

References · 44

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  4. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  5. 2017Robust Adversarial Reinforcement Learningcited by 439
  6. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  7. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  8. 2016Learning to reinforcement learncited by 369
  9. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 278
  10. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
  11. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  12. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
32 more
  1. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  2. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 228
  3. 2015Contextual Markov Decision Processescited by 213
  4. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 210
  5. 2015Actor-Mimic: Deep Multitask and Transfer Reinforcement Learningcited by 176
  6. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  7. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 167
  8. 2019Active Domain Randomizationcited by 161
  9. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 138
  10. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 136
  11. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 129
  12. 2014Optimizing the CVaR via Samplingcited by 98
  13. 2020One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RLcited by 79
  14. 2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 76
  15. 2019Meta reinforcement learning as task inferencecited by 75
  16. 2020Robust Reinforcement Learning using Adversarial Populationscited by 64
  17. 2017Reinforcement Learning under Model Mismatchcited by 52
  18. 2019Worst Cases Policy Gradientscited by 52
  19. 2020Rapidly Adaptable Legged Robots via Evolutionary Meta-Learningcited by 42
  20. 2020Cautious Adaptation For Reinforcement Learning in Safety-Critical Settingscited by 42
  21. 2018Policy Transfer with Strategy Optimizationcited by 41
  22. 2018Soft-Robust Actor-Critic Policy-Gradientcited by 37
  23. 2020Model-based Adversarial Meta-Reinforcement Learningcited by 32
  24. 2020Meta-Reinforcement Learning for Robotic Industrial Insertion Taskscited by 28
  25. 2020MetaCURE: Meta Reinforcement Learning with Empowerment-Driven Explorationcited by 24
  26. 2020MELD: Meta-Reinforcement Learning from Images via Latent State Modelscited by 22
  27. 2021Discovering a set of policies for the worst case rewardcited by 21
  28. 2018Bayesian Policy Optimization for Model Uncertaintycited by 12
  29. 2020Offline Meta Learning of Explorationcited by 12
  30. 2019Robust and Adaptive Planning under Model Uncertaintycited by 10
  31. 2020Model-Based Generalization Under Parameter Uncertainty Using Path Integral Controlcited by 7
  32. 2018Efficient transfer learning and online adaptation with latent variable models for continuous controlcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 5; 2024: 3; 2025: 2; 2026: 3.

16 papers.

Sort:

  1. 2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 30
  2. 2023Seeing is not Believing: Robust Reinforcement Learning against Spurious Correlationcited by 25
  3. 2024ODRL: A Benchmark for Off-Dynamics Reinforcement Learningcited by 25
  4. 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatchcited by 24
  5. 2022An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Contextcited by 15
  6. 2023Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackerscited by 15
  7. 2022Distributionally Adaptive Meta Reinforcement Learningcited by 11
  8. 2025Uncertainty Quantification for Machine Learning in Healthcare: A Surveycited by 11
  9. 2022Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variablescited by 6
  10. 2024GRAM: Generalization in Deep RL with a Robust Adaptation Modulecited by 2
  11. 2023Robustness to Multi-Modal Environment Uncertainty in MARL using Curriculum Learningcited by 1
  12. 2023Learning a Hierarchical Planner from Humans in Multiple Generationscited by 1
4 more
  1. 2025Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learningcited by 1
  2. 2026Cross-Domain Offline Policy Adaptation via Selective Transition Correctioncited by 1
  3. 2026Bayesian Robust Financial Trading with Adversarial Synthetic Market Datacited by 0
  4. 2026Robust Successor Featurescited by 0