mlsanityData to 7 Oct 2026

Solving the Real Robot Challenge using Deep Reinforcement Learning

arXiv 2109.15233 ↗First posted 30 Sep 2021cs.RO cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

This paper details our winning submission to Phase 1 of the 2021 Real Robot Challenge; a challenge in which a three-fingered robot must carry a cube along specified goal trajectories. To solve Phase 1, we use a pure reinforcement learning approach which requires minimal expert knowledge of the robotic system, or of robotic grasping in general. A sparse, goal-based reward is employed in conjunction with Hindsight Experience Replay to teach the control policy to move the cube to the desired x and y coordinates of the goal. Simultaneously, a dense distance-based reward is employed to teach the policy to lift the cube to the z coordinate (the height component) of the goal. The policy is trained in simulation with domain randomisation before being transferred to the real robot for evaluation. Although performance tends to worsen after this transfer, our best policy can successfully lift the real cube along goal trajectories via an effective pinching grasp. Our approach outperforms all other submissions, including those leveraging more traditional robotic control techniques, and is the first pure learning-based method to solve this challenge.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 23 of its 34 references that are in the map. First posted on arXiv 30 Sep 2021.

Builds on

23 of 34 references.

References · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  7. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  8. 2017Hindsight Experience Replaycited by 1,096
  9. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  10. 2019Solving Rubik's Cube with a Robot Handcited by 699
  11. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  12. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
11 more
  1. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  2. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  3. 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 200
  4. 2018Residual Policy Learningcited by 146
  5. 2020Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifierscited by 93
  6. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstrationcited by 92
  7. 2019Meta Reinforcement Learning for Sim-to-real Domain Adaptationcited by 49
  8. 2021Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Reviewcited by 34
  9. 2020TriFinger: An Open-Source Robot for Learning Dexteritycited by 31
  10. 2021Benchmarking Structured Policies and Policy Optimization for Real-World Dexterous Object Manipulationcited by 14
  11. 2021Grasp and Motion Planning for Dexterous Manipulation for the Real Robot Challengecited by 4

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 2; 2024: 1.

5 papers.

  1. 2023Improving Behavioural Cloning with Positive Unlabeled Learningcited by 8
  2. 2022Learning to Fold Real Garments with One Arm: A Case Study in Cloud-Based Robotics Researchcited by 6
  3. 2022Dexterous Robotic Manipulation using Deep Reinforcement Learning and Knowledge Transfer for Complex Sparse Reward-based Taskscited by 5
  4. 2023Identifying Expert Behavior in Offline Training Datasets Improves Behavioral Cloning of Robotic Manipulation Policiescited by 5
  5. 2024Dataset Clustering for Improved Offline Policy Learningcited by 1