# Learning Long-Horizon Robot Manipulation Skills via Privileged Action

arXiv: https://arxiv.org/abs/2502.15442 · HTML: https://mlsanity.com/paper/2502.15442/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 31 of its 41 references that are in the map.
- First posted on arXiv 21 Feb 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 30 of 31)

- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [MimicPlay: Long-Horizon Imitation Learning by Watching Human Play](https://mlsanity.com/paper/2302.12422/) (2023)
- [Rapid Locomotion via Reinforcement Learning](https://mlsanity.com/paper/2205.02824/) (2022)
- [A System for General In-Hand Object Re-Orientation](https://mlsanity.com/paper/2111.03043/) (2021)
- [Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition](https://mlsanity.com/paper/2307.14535/) (2023)
- [Robot Parkour Learning](https://mlsanity.com/paper/2309.05665/) (2023)
- [Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.13653/) (2023)
- [Learning High-Speed Flight in the Wild](https://mlsanity.com/paper/2110.05113/) (2021)
- [Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes](https://mlsanity.com/paper/2211.11744/) (2022)
- [General In-Hand Object Rotation with Vision and Touch](https://mlsanity.com/paper/2309.09979/) (2023)
- [Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Models](https://mlsanity.com/paper/2401.03360/) (2023)
- [Multi-Stage Cable Routing through Hierarchical Imitation Learning](https://mlsanity.com/paper/2307.08927/) (2023)
- [Object-Centric Dexterous Manipulation from Human Motion Data](https://mlsanity.com/paper/2411.04005/) (2024)
- [LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulation](https://mlsanity.com/paper/2210.12631/) (2022)
- [DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Training](https://mlsanity.com/paper/2305.12127/) (2023)
- [Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity](https://mlsanity.com/paper/2211.01500/) (2022)
- [Practice Makes Perfect: Planning to Learn Skill Parameter Policies](https://mlsanity.com/paper/2402.15025/) (2024)
- [AnyRotate: Gravity-Invariant In-Hand Object Rotation with Sim-to-Real Touch](https://mlsanity.com/paper/2405.07391/) (2024)
- [SAPG: Split and Aggregate Policy Gradients](https://mlsanity.com/paper/2407.20230/) (2024)
- [Pre- and post-contact policy decomposition for non-prehensile manipulation with zero-shot sim-to-real transfer](https://mlsanity.com/paper/2309.02754/) (2023)
- [The Art of Imitation: Learning Long-Horizon Manipulation Tasks from Few Demonstrations](https://mlsanity.com/paper/2407.13432/) (2024)
- [DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots](https://mlsanity.com/paper/2409.06613/) (2024)
- [SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation](https://mlsanity.com/paper/2410.18065/) (2024)
- [Enhancing Dexterity in Robotic Manipulation via Hierarchical Contact Exploration](https://mlsanity.com/paper/2307.00383/) (2023)
- [DexSkills: Skill Segmentation Using Haptic Data for Learning Autonomous Long-Horizon Robotic Manipulation Tasks](https://mlsanity.com/paper/2405.03476/) (2024)
- [Multi-Stage Reinforcement Learning for Non-Prehensile Manipulation](https://mlsanity.com/paper/2307.12074/) (2023)
- [Constrained Skill Discovery: Quadruped Locomotion with Unsupervised Reinforcement Learning](https://mlsanity.com/paper/2410.07877/) (2024)
- [Learning Visuotactile Estimation and Control for Non-prehensile Manipulation under Occlusions](https://mlsanity.com/paper/2412.13157/) (2024)

## Built on by (most cited first, 2 of 2)

- [DexMachina: Functional Retargeting for Bimanual Dexterous Manipulation](https://mlsanity.com/paper/2505.24853/) (2025)
- [VAMPS: Visual and Motor Policies from Sampling-Based Planning](https://mlsanity.com/paper/2610.05331/) (2026)

## Abstract (arXiv)

Long-horizon contact-rich tasks are challenging to learn with reinforcement learning, due to ineffective exploration of high-dimensional state spaces with sparse rewards. The learning process often gets stuck in local optimum and demands task-specific reward fine-tuning for complex scenarios. In this work, we propose a structured framework that leverages privileged actions with curriculum learning, enabling the policy to efficiently acquire long-horizon skills without relying on extensive reward engineering or reference trajectories. Specifically, we use privileged actions in simulation with a general training procedure that would be infeasible to implement in real-world scenarios. These privileges include relaxed constraints and virtual forces that enhance interaction and exploration with objects. Our results successfully achieve complex multi-stage long-horizon tasks that naturally combine non-prehensile manipulation with grasping to lift objects from non-graspable poses. We demonstrate generality by maintaining a parsimonious reward structure and showing convergence to diverse and robust behaviors across various environments. Additionally, real-world experiments further confirm that the skills acquired using our approach are transferable to real-world environments, exhibiting robust and intricate performance. Our approach outperforms state-of-the-art methods in these tasks, converging to solutions where others fail.