mlsanityData to 14 Oct 2026

End-to-End Robotic Reinforcement Learning without Reward Engineering

arXiv 1904.07854 ↗First posted 16 Apr 2019cs.LG cs.CV cs.ROcited by 122

Not on the map. Why?

Cited by 33 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The combination of deep neural network models and reinforcement learning algorithms can make it possible to learn policies for robotic behaviors that directly read in raw sensory inputs, such as camera images, effectively subsuming both estimation and control into one model. However, real-world applications of reinforcement learning must specify the goal of the task by means of a manually programmed reward function, which in practice requires either designing the very same perception pipeline that end-to-end reinforcement learning promises to avoid, or else instrumenting the environment with additional sensors to determine if the task has been performed successfully. In this paper, we propose an approach for removing the need for manual engineering of reward specifications by enabling a robot to learn from a modest number of examples of successful outcomes, followed by actively solicited queries, where the robot shows the user a state and asks for a label to determine whether that state represents successful completion of the task. While requesting labels for every single state would amount to asking the user to manually provide the reward signal, our method requires labels for only a tiny fraction of the states seen during training, making it an efficient and practical approach for learning skills without manually engineered rewards. We evaluate our method on real-world robotic manipulation tasks where the observations consist of images viewed by the robot's camera. In our experiments, our method effectively learns to arrange objects, place books, and drape cloth, directly from images and without any manually specified reward functions, and with only 1-4 hours of interaction with the real world.

Key facts. Cited by 122 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 4; 2020: 6; 2021: 26; 2022: 27; 2023: 22; 2024: 20; 2025: 8; 2026: 9.

122 papers.

Sort:

By area:

  1. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  2. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 395
  3. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 302
  4. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  5. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 209
  6. 2023Vision-Language Models as Success Detectorscited by 142
  7. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 119
  8. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 118
  9. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 116
  10. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperationcited by 112
  11. 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 107
  12. 2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 95
38 more
  1. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstrationcited by 93
  2. 2020COG: Connecting New Skills to Past Experience with Offline Reinforcement Learningcited by 83
  3. 2022Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trialscited by 78
  4. 2022Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation?cited by 70
  5. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 70
  6. 2022How to Leverage Unlabeled Data in Offline Reinforcement Learningcited by 62
  7. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 62
  8. 2025RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learningcited by 59
  9. 2022Holo-Dex: Teaching Dexterity with Immersive Mixed Realitycited by 57
  10. 2023Diffusion Reward: Learning Rewards via Conditional Video Diffusioncited by 57
  11. 2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 54
  12. 2022Invariance in Policy Optimisation and Partial Identifiability in Reward Learningcited by 46
  13. 2019Task-Relevant Adversarial Imitation Learningcited by 45
  14. 2021Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classificationcited by 38
  15. 2024Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applicationscited by 38
  16. 2021Lifelong Robotic Reinforcement Learning by Retaining Experiencescited by 34
  17. 2023Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learningcited by 34
  18. 2024Rank2Reward: Learning Shaped Reward Functions from Passive Videocited by 32
  19. 2022Affordance Learning from Play for Sample-Efficient Policy Learningcited by 30
  20. 2021A Max-Min Entropy Framework for Reinforcement Learningcited by 28
  21. 2021Model-Based Reinforcement Learning via Latent-Space Collocationcited by 28
  22. 2022Misspecification in Inverse Reinforcement Learningcited by 27
  23. 2021Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racingcited by 26
  24. 2024Leveraging Symmetry in RL-based Legged Locomotion Controlcited by 25
  25. 2021MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven Reinforcement Learningcited by 24
  26. 2022You Only Live Once: Single-Life Reinforcement Learningcited by 24
  27. 2020Semi-supervised reward learning for offline reinforcement learningcited by 23
  28. 2022Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidancecited by 23
  29. 2024Affordance-Guided Reinforcement Learning via Visual Promptingcited by 23
  30. 2022A State-Distribution Matching Approach to Non-Episodic Reinforcement Learningcited by 22
  31. 2024Continuously Improving Mobile Manipulation with Autonomous Real-World RLcited by 22
  32. 2020Scalable Multi-Task Imitation Learning with Autonomous Improvementcited by 21
  33. 2019Scaling data-driven robotics with reward sketching and batch reinforcement learningcited by 18
  34. 2023MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observationscited by 18
  35. 2021Outcome-Driven Reinforcement Learning via Variational Inferencecited by 17
  36. 2021Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learningcited by 15
  37. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 15
  38. 2021Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Taskscited by 13

50 most cited of 122 All 122 (JSON)