mlsanityData to 14 Oct 2026

Robot Trains Robot: Automatic Real-World Policy Adaptation and Learning for Humanoids

arXiv 2508.12252 ↗First posted 17 Aug 2025cs.ROcited by 7

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Simulation-based reinforcement learning (RL) has significantly advanced humanoid locomotion tasks, yet direct real-world RL from scratch or adapting from pretrained policies remains rare, limiting the full potential of humanoid robots. Real-world learning, despite being crucial for overcoming the sim-to-real gap, faces substantial challenges related to safety, reward design, and learning efficiency. To address these limitations, we propose Robot-Trains-Robot (RTR), a novel framework where a robotic arm teacher actively supports and guides a humanoid robot student. The RTR system provides protection, learning schedule, reward, perturbation, failure detection, and automatic resets. It enables efficient long-term real-world humanoid training with minimal human intervention. Furthermore, we propose a novel RL pipeline that facilitates and stabilizes sim-to-real transfer by optimizing a single dynamics-encoded latent variable in the real world. We validate our method through two challenging real-world humanoid tasks: fine-tuning a walking policy for precise speed tracking and learning a humanoid swing-up task from scratch, illustrating the promising capabilities of real-world humanoid learning realized by RTR-style systems. See https://robot-trains-robot.github.io/ for more info.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 49 of its 54 references that are in the map. First posted on arXiv 17 Aug 2025.

Builds on

49 of 54 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids27 Feb
  2. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning29 Oct
  3. 2024MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learning19 Oct
  4. 2024Advancing Humanoid Locomotion: Mastering Challenging Terrains with Denoising World Model Learning26 Aug
  5. 2024Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learning22 Jul

Other references · 44

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  4. 2019Solving Rubik's Cube with a Robot Handcited by 734
  5. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  6. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  7. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  8. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  9. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  10. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
  11. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 251
  12. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 217
32 more
  1. 2023Real-World Humanoid Locomotion with Reinforcement Learningcited by 187
  2. 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 185
  3. 2023Extreme Parkour with Legged Robotscited by 164
  4. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
  5. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 130
  6. 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 130
  7. 2022In-Hand Object Rotation via Rapid Motor Adaptationcited by 118
  8. 2020Learning to Walk in the Real World with Minimal Human Effortcited by 111
  9. 2022Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapescited by 101
  10. 2024Humanoid Parkour Learningcited by 96
  11. 2024HOVER: Versatile Neural Whole-Body Controller for Humanoid Robotscited by 91
  12. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 83
  13. 2024UMI on Legs: Making Manipulation Policies Mobile with Manipulation-Centric Whole-body Controllerscited by 75
  14. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 59
  15. 2019Sim-to-Real Transfer for Biped Locomotioncited by 56
  16. 2022Adapting Rapid Motor Adaptation for Bipedal Robotscited by 54
  17. 2023Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulationcited by 49
  18. 2019Learning Fast Adaptation with Meta Strategy Optimizationcited by 48
  19. 2024Adaptive Compliance Policy: Learning Approximate Compliance for Diffusion Guided Controlcited by 41
  20. 2024Object-Centric Dexterous Manipulation from Human Motion Datacited by 41
  21. 2020Meta-Reinforcement Learning for Robotic Industrial Insertion Taskscited by 34
  22. 2024Adaptive Mobile Manipulation for Articulated Objects In the Open Worldcited by 31
  23. 2023What Went Wrong? Closing the Sim-to-Real Gap via Differentiable Causal Discoverycited by 27
  24. 2023Uni-O4: Unifying Online and Offline Deep Reinforcement Learning with Multi-Step On-Policy Optimizationcited by 26
  25. 2024Learning Humanoid Locomotion over Challenging Terraincited by 26
  26. 2023Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotioncited by 24
  27. 2022Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidancecited by 23
  28. 2024Continuously Improving Mobile Manipulation with Autonomous Real-World RLcited by 22
  29. 2023AdaptSim: Task-Driven Simulation Adaptation for Sim-to-Real Transfercited by 18
  30. 2024Make-An-Agent: A Generalizable Policy Network Generator with Behavior-Prompted Diffusioncited by 14
  31. 2023Cherry-Picking with Reinforcement Learning : Robust Dynamic Grasping in Unstable Conditionscited by 11
  32. 2025ToddlerBot: Open-Source ML-Compatible Humanoid Platform for Loco-Manipulationcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Towards Bridging the Gap between Large-Scale Pretraining and Efficient Finetuning for Humanoid Controlcited by 1
  2. 2026One Demonstration Is Enough for Real-World Robotic Reinforcement Learningcited by 1
  3. 2026Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Modelscited by 1
  4. 2026HandelBot: Real-World Piano Playing via Fast Adaptation of Dexterous Robot Policiescited by 0
  5. 2026Learning Humanoid Navigation from Human Datacited by 0
  6. 2026Dynamics Are Learned, Not Told: Semi-Supervised Discovery of Latent Dynamics Geometries For Zero-Shot Policy Adaptationcited by 0
  7. 2026From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Interventioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning Sim-to-Real Humanoid Locomotion in 15 Minutes
  2. 2024HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation
  3. 2025FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Control
  4. 2024Learn to Teach: Sample-Efficient Privileged Learning for Humanoid Locomotion over Diverse Terrains
  5. 2026AGILE: A Comprehensive Workflow for Humanoid Loco-Manipulation Learning
  6. 2025Booster Gym: An End-to-End Reinforcement Learning Framework for Humanoid Robot Locomotion
  7. 2025Humanoid Goalkeeper: Learning from Position Conditioned Task-Motion Constraints
  8. 2026HuMBLE: Human Motion-Driven Behavior Learning for Embodied Locomotion ↗