mlsanityData to 7 Oct 2026

Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deployment

arXiv 2311.01059 ↗First posted 2 Nov 2023cs.RO cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

To succeed in the real world, robots must cope with situations that differ from those seen during training. We study the problem of adapting on-the-fly to such novel scenarios during deployment, by drawing upon a diverse repertoire of previouslylearned behaviors. Our approach, RObust Autonomous Modulation (ROAM), introduces a mechanism based on the perceived value of pre-trained behaviors to select and adapt pre-trained behaviors to the situation at hand. Crucially, this adaptation process all happens within a single episode at test time, without any human supervision. We demonstrate that ROAM enables a robot to adapt rapidly to changes in dynamics both in simulation and on a real Go1 quadruped, even successfully moving forward with roller skates on its feet. Our approach adapts over 2x as efficiently compared to existing methods when facing a variety of out-of-distribution situations during deployment by effectively choosing and adapting relevant behaviors on-the-fly.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 66 of its 72 references that are in the map. First posted on arXiv 2 Nov 2023.

Builds on

66 of 72 references.

References · 66

  1. 2016Layer Normalizationcited by 5,579
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  6. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  7. 2016Progressive Neural Networkscited by 1,332
  8. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  9. 2019Solving Rubik's Cube with a Robot Handcited by 699
  10. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  11. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  12. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
38 more
  1. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  2. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  3. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  4. 2016The Option-Critic Architecturecited by 433
  5. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  6. 2016Learning to reinforcement learncited by 369
  7. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
  8. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
  9. 2023Efficient Online Reinforcement Learning with Offline Datacited by 332
  10. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  11. 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 299
  12. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
  13. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  14. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
  15. 2016Variational Intrinsic Controlcited by 262
  16. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 229
  17. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  18. 2014Robots that can adapt like animalscited by 203
  19. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 136
  20. 2017On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learningcited by 136
  21. 2022Legged Locomotion in Challenging Terrains using Egocentric Visioncited by 131
  22. 2022Rapid Locomotion via Reinforcement Learningcited by 130
  23. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 129
  24. 2018Variational Option Discovery Algorithmscited by 121
  25. 2020Self-Supervised Policy Adaptation during Deploymentcited by 121
  26. 2023Robot Parkour Learningcited by 115
  27. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
  28. 2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 108
  29. 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 107
  30. 2019MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policiescited by 105
  31. 2022Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotioncited by 98
  32. 2020Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifierscited by 93
  33. 2018ProMP: Proximal Meta-Policy Searchcited by 91
  34. 2018Evolved Policy Gradientscited by 86
  35. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 76
  36. 2021Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Taskscited by 75
  37. 2021Demonstration-Guided Reinforcement Learning with Learned Skillscited by 73
  38. 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 66

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 2.

7 papers.

  1. 2024Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Modelscited by 15
  2. 2025Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoningcited by 10
  3. 2024GRAM: Generalization in Deep RL with a Robust Adaptation Modulecited by 2
  4. 2026RACaP: Agentic Reasoning, Acting, and Coding as Policies for Evolvable Robot Learningcited by 1
  5. 2024Enhancing Hardware Fault Tolerance in Machines with Reinforcement Learning Policy Gradient Algorithmscited by 0
  6. 2025Requirements for Recognition and Rapid Response to Unfamiliar Events Outside of Agent Design Scopecited by 0
  7. 2026WINFlowNets: Warm-up Integrated Networks Training of Generative Flow Networks for Robotics and Machine Fault Adaptationcited by 0