# Reinforcement Learning for Robotic Manipulation using Simulated Locomotion Demonstrations

arXiv: https://arxiv.org/abs/1910.07294 · HTML: https://mlsanity.com/paper/1910.07294/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2019.

## Built on by (most cited first, 11 of 11)

- [SEIL: Simulation-augmented Equivariant Imitation Learning](https://mlsanity.com/paper/2211.00194/) (2022)
- [Rethinking Adversarial Attacks in Reinforcement Learning from Policy Distribution Perspective](https://mlsanity.com/paper/2501.03562/) (2025)
- [VARP: Reinforcement Learning from Vision-Language Model Feedback with Agent Regularized Preferences](https://mlsanity.com/paper/2503.13817/) (2025)
- [Imitate the Good and Avoid the Bad: An Incremental Approach to Safe Reinforcement Learning](https://mlsanity.com/paper/2312.10385/) (2023)
- [SPRINQL: Sub-optimal Demonstrations driven Offline Imitation Learning](https://mlsanity.com/paper/2402.13147/) (2024)
- [Unsupervised Skill Discovery as Exploration for Learning Agile Locomotion](https://mlsanity.com/paper/2508.08982/) (2025)
- [Hierarchical Primitive Composition: Simultaneous Activation of Skills with Inconsistent Action Dimensions in Multiple Hierarchies](https://mlsanity.com/paper/2110.01833/) (2021)
- [Dealing with Sparse Rewards in Continuous Control Robotics via Heavy-Tailed Policies](https://mlsanity.com/paper/2206.05652/) (2022)
- [Simultaneously Updating All Persistence Values in Reinforcement Learning](https://mlsanity.com/paper/2211.11620/) (2022)
- [Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPO](https://mlsanity.com/paper/2606.02313/) (2026)
- [Hybrid Robot Learning for Automatic Robot Motion Planning in Manufacturing](https://mlsanity.com/paper/2502.19340/) (2025)

## Abstract (arXiv)

Mastering robotic manipulation skills through reinforcement learning (RL) typically requires the design of shaped reward functions. Recent developments in this area have demonstrated that using sparse rewards, i.e. rewarding the agent only when the task has been successfully completed, can lead to better policies. However, state-action space exploration is more difficult in this case. Recent RL approaches to learning with sparse rewards have leveraged high-quality human demonstrations for the task, but these can be costly, time consuming or even impossible to obtain. In this paper, we propose a novel and effective approach that does not require human demonstrations. We observe that every robotic manipulation task could be seen as involving a locomotion task from the perspective of the object being manipulated, i.e. the object could learn how to reach a target state on its own. In order to exploit this idea, we introduce a framework whereby an object locomotion policy is initially obtained using a realistic physics simulator. This policy is then used to generate auxiliary rewards, called simulated locomotion demonstration rewards (SLDRs), which enable us to learn the robot manipulation policy. The proposed approach has been evaluated on 13 tasks of increasing complexity, and can achieve higher success rate and faster learning rates compared to alternative algorithms. SLDRs are especially beneficial for tasks like multi-object stacking and non-rigid object manipulation.