mlsanityData to 7 Oct 2026

The Ingredients of Real-World Robotic Reinforcement Learning

arXiv 2004.12570 ↗First posted 27 Apr 2020cs.LG cs.RO stat.MLcited by 111

Not on the map. Why?

Abstract (arXiv)

The success of reinforcement learning for real world robotics has been, in many cases limited to instrumented laboratory scenarios, often requiring arduous human effort and oversight to enable continuous learning. In this work, we discuss the elements that are needed for a robotic learning system that can continually and autonomously improve with data collected in the real world. We propose a particular instantiation of such a system, using dexterous manipulation as our case study. Subsequently, we investigate a number of challenges that come up when learning without instrumentation. In such settings, learning must be feasible without manually designed resets, using only on-board perception, and without hand-engineered reward functions. We propose simple and scalable solutions to these challenges, and then demonstrate the efficacy of our proposed system on a set of dexterous robotic manipulation tasks, providing an in-depth analysis of the challenges associated with this learning paradigm. We demonstrate that our complete system can learn without any human intervention, acquiring a variety of vision-based skills with a real-world three-fingered hand. Results and videos can be found at https://sites.google.com/view/realworld-rl/

Key facts. Cited by 111 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 18; 2022: 19; 2023: 20; 2024: 22; 2025: 16; 2026: 15.

111 papers.

Sort:

By area:

  1. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  2. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  3. 2022Human-to-Robot Imitation in the Wildcited by 157
  4. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  5. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policycited by 139
  6. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 135
  7. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  8. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 113
  9. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 90
  10. 2022Few-Shot Preference Learning for Human-in-the-Loop RLcited by 85
  11. 2023Inverse Preference Learning: Preference-based RL without a Reward Functioncited by 68
  12. 2023Imitating Task and Motion Planning with Visuomotor Transformerscited by 63
38 more
  1. 2024FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-Tuningcited by 54
  2. 2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 49
  3. 2026RL Token: Bootstrapping Online RL with Vision-Language-Action Modelscited by 43
  4. 2023Learning Zero-Shot Cooperation with Humans, Assuming Humans Are Biasedcited by 37
  5. 2022GenLoco: Generalized Locomotion Controllers for Quadrupedal Robotscited by 36
  6. 2021Autonomous Reinforcement Learning: Formalism and Benchmarkingcited by 33
  7. 2021Autonomous Reinforcement Learning via Subgoal Curriculacited by 32
  8. 2022Stabilizing Off-Policy Deep Reinforcement Learning from Pixelscited by 32
  9. 2024Autonomous Improvement of Instruction Following Skills via Foundation Modelscited by 32
  10. 2021Fully Autonomous Real-World Reinforcement Learning with Applications to Mobile Manipulationcited by 30
  11. 2023Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learningcited by 29
  12. 2022Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learningcited by 28
  13. 2025Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstrationcited by 27
  14. 2021MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven Reinforcement Learningcited by 24
  15. 2022You Only Live Once: Single-Life Reinforcement Learningcited by 23
  16. 2021Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racingcited by 22
  17. 2022A State-Distribution Matching Approach to Non-Episodic Reinforcement Learningcited by 21
  18. 2023Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotioncited by 21
  19. 2022When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learningcited by 20
  20. 2022Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidancecited by 20
  21. 2024Uni-RLHF: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedbackcited by 20
  22. 2024Continuously Improving Mobile Manipulation with Autonomous Real-World RLcited by 20
  23. 2024MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learningcited by 20
  24. 2022Asynchronous Reinforcement Learning for Real-Time Control of Physical Robotscited by 19
  25. 2024So You Think You Can Scale Up Autonomous Robot Data Collection?cited by 17
  26. 2022Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Realcited by 16
  27. 2023FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicingcited by 13
  28. 2026Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learningcited by 13
  29. 2021Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Taskscited by 12
  30. 2023REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulationcited by 12
  31. 2023Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedbackcited by 12
  32. 2022Automating Reinforcement Learning with Example-based Resetscited by 11
  33. 2023Efficient RL via Disentangled Environment and Agent Representationscited by 11
  34. 2026Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language Modelscited by 11
  35. 2021Learning to Centralize Dual-Arm Assemblycited by 9
  36. 2023DRAGON: A Dialogue-Based Robot for Assistive Navigation with Visual Language Groundingcited by 9
  37. 2024Nearly Optimal Algorithms for Contextual Dueling Bandits from Adversarial Feedbackcited by 9
  38. 2024xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editingcited by 9

50 most cited of 111 All 111 (JSON)