# Setting up a Reinforcement Learning Task with a Real-World Robot

arXiv: https://arxiv.org/abs/1803.07067 · HTML: https://mlsanity.com/paper/1803.07067/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2018.

## Built on by (most cited first, 30 of 33)

- [Near-Optimal Regret for Adversarial MDP with Delayed Bandit Feedback](https://mlsanity.com/paper/2201.13172/) (2022)
- [Asynchronous Reinforcement Learning for Real-Time Control of Physical Robots](https://mlsanity.com/paper/2203.12759/) (2022)
- [Delayed Reinforcement Learning by Imitation](https://mlsanity.com/paper/2205.05569/) (2022)
- [Real-Time Reinforcement Learning for Vision-Based Robotics Utilizing Local and Remote Computers](https://mlsanity.com/paper/2210.02317/) (2022)
- [Dynamic Decision Frequency with Continuous Options](https://mlsanity.com/paper/2212.04407/) (2022)
- [Robustness and Cybersecurity in the EU Artificial Intelligence Act](https://mlsanity.com/paper/2502.16184/) (2025)
- [Boosting Reinforcement Learning with Strongly Delayed Feedback Through Auxiliary Short Delays](https://mlsanity.com/paper/2402.03141/) (2024)
- [Variational Delayed Policy Optimization](https://mlsanity.com/paper/2405.14226/) (2024)
- [Reinforcement Learning for Blind Stair Climbing with Legged and Wheeled-Legged Robots](https://mlsanity.com/paper/2402.06143/) (2024)
- [A Reduction-based Framework for Sequential Decision Making with Delayed Feedback](https://mlsanity.com/paper/2302.01477/) (2023)
- [DiAReL: Reinforcement Learning with Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Control](https://mlsanity.com/paper/2306.09010/) (2023)
- [Delay-Adapted Policy Optimization and Improved Regret for Adversarial MDP with Delayed Bandit Feedback](https://mlsanity.com/paper/2305.07911/) (2023)
- [RMBench: Benchmarking Deep Reinforcement Learning for Robotic Manipulator Control](https://mlsanity.com/paper/2210.11262/) (2022)
- [Deployable Reinforcement Learning with Variable Control Rate](https://mlsanity.com/paper/2401.09286/) (2024)
- [Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning](https://mlsanity.com/paper/2406.03276/) (2024)
- [A Prescriptive Dirichlet Power Allocation Policy with Deep Reinforcement Learning](https://mlsanity.com/paper/2201.08445/) (2022)
- [Model-Based Reinforcement Learning under Random Observation Delays](https://mlsanity.com/paper/2509.20869/) (2025)
- [What Robot do I Need? Fast Co-Adaptation of Morphology and Control using Graph Neural Networks](https://mlsanity.com/paper/2111.02371/) (2021)
- [Tree Search-Based Policy Optimization under Stochastic Execution Delay](https://mlsanity.com/paper/2404.05440/) (2024)
- [An Idiosyncrasy of Time-discretization in Reinforcement Learning](https://mlsanity.com/paper/2406.14951/) (2024)
- [Performance Variation in Deep Reinforcement Learning](https://mlsanity.com/paper/2606.06746/) (2026)
- [A Concise Introduction to Reinforcement Learning in Robotics](https://mlsanity.com/paper/2210.07397/) (2022)
- [Control in Stochastic Environment with Delays: A Model-based Reinforcement Learning Approach](https://mlsanity.com/paper/2402.00313/) (2024)
- [Belief-Based Offline Reinforcement Learning for Delay-Robust Policy Optimization](https://mlsanity.com/paper/2506.00131/) (2025)
- [Reinforcement Learning via Conservative Agent for Environments with Random Delays](https://mlsanity.com/paper/2507.18992/) (2025)
- [Benchmarking Action Spaces in Reinforcement Learning for Vision-based Robotic Manipulation](https://mlsanity.com/paper/2606.18594/) (2026)
- [Physical Atari: A Robust and Accessible Platform for Real-time Reinforcement Learning on Robots](https://mlsanity.com/paper/2606.19357/) (2026)
- [Learning Visual Tracking and Reaching with Deep Reinforcement Learning on a UR10e Robotic Arm](https://mlsanity.com/paper/2308.14652/) (2023)
- [Inverse Delayed Reinforcement Learning](https://mlsanity.com/paper/2412.02931/) (2024)
- [Delayed homomorphic reinforcement learning for environments with delayed feedback](https://mlsanity.com/paper/2604.03641/) (2026)

## Abstract (arXiv)

Reinforcement learning is a promising approach to developing hard-to-engineer adaptive solutions for complex and diverse robotic tasks. However, learning with real-world robots is often unreliable and difficult, which resulted in their low adoption in reinforcement learning research. This difficulty is worsened by the lack of guidelines for setting up learning tasks with robots. In this work, we develop a learning task with a UR5 robotic arm to bring to light some key elements of a task setup and study their contributions to the challenges with robots. We find that learning performance can be highly sensitive to the setup, and thus oversights and omissions in setup details can make effective learning, reproducibility, and fair comparison hard. Our study suggests some mitigating steps to help future experimenters avoid difficulties and pitfalls. We show that highly reliable and repeatable experiments can be performed in our setup, indicating the possibility of reinforcement learning research extensively based on real-world robots.