# Off-Policy Deep Reinforcement Learning Algorithms for Handling Various Robotic Manipulator Tasks

arXiv: https://arxiv.org/abs/2212.05572 · HTML: https://mlsanity.com/paper/2212.05572/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 42 references that are in the map.
- First posted on arXiv 11 Dec 2022.

## Builds on (most cited first, 19 of 19)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Overcoming Exploration in Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/1709.10089/) (2017)
- [Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation](https://mlsanity.com/paper/1708.05144/) (2017)
- [A Survey on Model-based Reinforcement Learning](https://mlsanity.com/paper/2206.09328/) (2022)
- [Breaking the Sample Complexity Barrier to Regret-Optimal Model-Free Reinforcement Learning](https://mlsanity.com/paper/2110.04645/) (2021)
- [Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulation](https://mlsanity.com/paper/1904.06703/) (2019)
- [Learning to Play Table Tennis From Scratch using Muscular Robots](https://mlsanity.com/paper/2006.05935/) (2020)
- [An Off-policy Policy Gradient Theorem Using Emphatic Weightings](https://mlsanity.com/paper/1811.09013/) (2018)
- [Deep Deterministic Policy Gradient for Urban Traffic Light Control](https://mlsanity.com/paper/1703.09035/) (2017)
- [Robotic Table Tennis with Model-Free Reinforcement Learning](https://mlsanity.com/paper/2003.14398/) (2020)
- [Reinforcement Learning Experiments and Benchmark for Solving Robotic Reaching Tasks](https://mlsanity.com/paper/2011.05782/) (2020)
- [Bipedal Walking Robot using Deep Deterministic Policy Gradient](https://mlsanity.com/paper/1807.05924/) (2018)
- [Deep Learning with Experience Ranking Convolutional Neural Network for Robot Manipulator](https://mlsanity.com/paper/1809.05819/) (2018)

## Built on by (most cited first, 1 of 1)

- [A Reinforcement Learning Approach to Non-prehensile Manipulation through Sliding](https://mlsanity.com/paper/2502.17221/) (2025)

## Abstract (arXiv)

In order to avoid conventional controlling methods which created obstacles due to the complexity of systems and intense demand on data density, developing modern and more efficient control methods are required. In this way, reinforcement learning off-policy and model-free algorithms help to avoid working with complex models. In terms of speed and accuracy, they become prominent methods because the algorithms use their past experience to learn the optimal policies. In this study, three reinforcement learning algorithms; DDPG, TD3 and SAC have been used to train Fetch robotic manipulator for four different tasks in MuJoCo simulation environment. All of these algorithms are off-policy and able to achieve their desired target by optimizing both policy and value functions. In the current study, the efficiency and the speed of these three algorithms are analyzed in a controlled environment.