GRAM: Generalization in Deep RL with a Robust Adaptation Module
On the map: Deep reinforcement learning methods → Deep RL algorithms and representations
Abstract (arXiv)
The reliable deployment of deep reinforcement learning in real-world settings requires the ability to generalize across a variety of conditions, including both in-distribution scenarios seen during training as well as novel out-of-distribution scenarios. In this work, we present a framework for dynamics generalization in deep reinforcement learning that unifies these two distinct types of generalization within a single architecture. We introduce a robust adaptation module that provides a mechanism for identifying and reacting to both in-distribution and out-of-distribution environment dynamics, along with a joint training pipeline that combines the goals of in-distribution adaptation and out-of-distribution robustness. Our algorithm GRAM achieves strong generalization performance across in-distribution and out-of-distribution scenarios upon deployment, which we demonstrate through extensive simulation and hardware locomotion experiments on a quadruped robot.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 43 of its 49 references that are in the map. First posted on arXiv 5 Dec 2024.
Builds on
43 of 49 references.
References · 43
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
- 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
- 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
- 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
- 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
- 2017Robust Adversarial Reinforcement Learningcited by 451
- 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
- 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 295
- 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 276
- 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 238
- 2015Contextual Markov Decision Processescited by 220
31 more
- 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
- 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
- 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 174
- 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 170
- 2022Rapid Locomotion via Reinforcement Learningcited by 168
- 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 138
- 2022Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behaviorcited by 138
- 2021Epistemic Neural Networkscited by 120
- 2020Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learningcited by 98
- 2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 78
- 2024Agile But Safe: Learning Collision-Free High-Speed Legged Locomotioncited by 73
- 2021Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimationcited by 71
- 2024Real-Time Anomaly Detection and Reactive Planning with Large Language Modelscited by 68
- 2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 47
- 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 46
- 2023Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Responsecited by 39
- 2020Model-based Adversarial Meta-Reinforcement Learningcited by 33
- 2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 20
- 2019Single Episode Policy Transfer in Reinforcement Learningcited by 18
- 2022Robust Policy Learning over Multiple Uncertainty Setscited by 18
- 2024Dreaming of Many Worlds: Learning Contextual World Models Aids Zero-Shot Generalizationcited by 16
- 2022An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Contextcited by 15
- 2024Rethinking Robustness Assessment: Adversarial Attacks on Learning-based Quadrupedal Locomotion Controllerscited by 14
- 2023Risk-Averse Model Uncertainty for Distributionally Robust Safe Reinforcement Learningcited by 12
- 2022Distributionally Adaptive Meta Reinforcement Learningcited by 11
- 2023Robust Adversarial Reinforcement Learning via Bounded Rationality Curriculacited by 10
- 2022Reinforcement Learning in Presence of Discrete Markovian Context Evolutioncited by 9
- 2023Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guaranteescited by 9
- 2023Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deploymentcited by 8
- 2024PA-LOCO: Learning Perturbation-Adaptive Locomotion for Quadruped Robotscited by 8
- 2024Safe, Out-of-Distribution-Adaptive MPC with Conformalized Neural Network Ensemblescited by 3
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2024A Survey Analyzing Generalization in Deep Reinforcement Learning
- 2026Robust Successor Features
- 2024Improving Generalization on the ProcGen Benchmark with Simple Architectural Changes and Scale
- 2022Domain Generalization for Robust Model-Based Offline Reinforcement Learning
- 2026Paying Less Generalization Tax: A Cross-Domain Generalization Study of RL Training for LLM Agents
- 2024Doubly Mild Generalization for Offline Reinforcement Learning
- 2025Robust Finetuning of Vision-Language-Action Robot Policies via Parameter Merging
- 2026Many Ways to Succeed: Diversity-Driven RL Fine-Tuning for VLA Generalization