mlsanityData to 7 Oct 2026

Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning

arXiv 1803.11347 ↗First posted 30 Mar 2018cs.LG cs.RO stat.MLcited by 270

Not on the map. Why?

Cited by 21 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Although reinforcement learning methods can achieve impressive results in simulation, the real world presents two major challenges: generating samples is exceedingly expensive, and unexpected perturbations or unseen situations cause proficient but specialized policies to fail at test time. Given that it is impractical to train separate policies to accommodate all situations the agent may see in the real world, this work proposes to learn how to quickly and effectively adapt online to new tasks. To enable sample-efficient learning, we consider learning online adaptation in the context of model-based reinforcement learning. Our approach uses meta-learning to train a dynamics model prior such that, when combined with recent data, this prior can be rapidly adapted to the local context. Our experiments demonstrate online adaptation for continuous control tasks on both simulated and real-world agents. We first show simulated agents adapting their behavior online to novel terrains, crippled body parts, and highly-dynamic environments. We also illustrate the importance of incorporating online adaptation into autonomous agents that operate in the real world by applying our method to a real dynamic legged millirobot. We demonstrate the agent's learned ability to quickly adapt online to a missing leg, adjust to novel terrains and slopes, account for miscalibration or errors in pose estimation, and compensate for pulling payloads.

Key facts. Cited by 270 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 1; 2020: 2; 2021: 62; 2022: 53; 2023: 47; 2024: 44; 2025: 37; 2026: 23.

270 papers.

Sort:

By area:

  1. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  2. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 240
  3. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  4. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  5. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 137
  6. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 136
  7. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agentscited by 135
  8. 2022Legged Locomotion in Challenging Terrains using Egocentric Visioncited by 131
  9. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 118
  10. 2023Robot Parkour Learningcited by 115
  11. 2022Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotioncited by 98
  12. 2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 92
38 more
  1. 2022Few-Shot Preference Learning for Human-in-the-Loop RLcited by 85
  2. 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 82
  3. 2022Neural-Fly Enables Rapid Learning for Agile Flight in Strong Windscited by 80
  4. 2022MyoSuite -- A contact-rich simulation suite for musculoskeletal motor controlcited by 80
  5. 2021Continual Backprop: Stochastic Gradient Descent with Persistent Randomnesscited by 79
  6. 2024DrEureka: Language Model Guided Sim-To-Real Transfercited by 78
  7. 2022A Survey on Model-based Reinforcement Learningcited by 65
  8. 2021Robot Learning from Randomized Simulations: A Reviewcited by 56
  9. 2023How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligencecited by 55
  10. 2022Generalizing to New Physical Systems via Context-Informed Dynamics Modelcited by 49
  11. 2021AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learningcited by 46
  12. 2021Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learningcited by 46
  13. 2021Coupling Vision and Proprioception for Navigation of Legged Robotscited by 46
  14. 2023Maintaining Plasticity in Deep Continual Learningcited by 42
  15. 2021Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environmentcited by 41
  16. 2021PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learningcited by 41
  17. 2022Adapting Rapid Motor Adaptation for Bipedal Robotscited by 39
  18. 2021Adaptive-Control-Oriented Meta-Learning for Nonlinear Systemscited by 38
  19. 2022Skill-based Meta-Reinforcement Learningcited by 36
  20. 2021Deep reinforcement learning in medical imaging: A literature reviewcited by 34
  21. 2021Lifelong Robotic Reinforcement Learning by Retaining Experiencescited by 34
  22. 2021Context-Aware Safe Reinforcement Learning for Non-Stationary Environmentscited by 33
  23. 2022A System-Level View on Out-of-Distribution Data in Roboticscited by 33
  24. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  25. 2021Procedural Generalization by Planning with Self-Supervised World Modelscited by 30
  26. 2022Factored Adaptation for Non-Stationary Reinforcement Learningcited by 30
  27. 2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 30
  28. 2019Watch, Try, Learn: Meta-Learning from Demonstrations and Rewardcited by 26
  29. 2021Block Contextual MDPs for Continual Learningcited by 25
  30. 2024ODRL: A Benchmark for Off-Dynamics Reinforcement Learningcited by 25
  31. 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatchcited by 24
  32. 2021Meta-Adaptive Nonlinear Control: Theory and Algorithmscited by 23
  33. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23
  34. 2022You Only Live Once: Single-Life Reinforcement Learningcited by 23
  35. 2021Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platformscited by 22
  36. 2021How to Train Your MAML to Excel in Few-Shot Classificationcited by 22
  37. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 22
  38. 2022Interactron: Embodied Adaptive Object Detectioncited by 21

50 most cited of 270 All 270 (JSON)