# Learning Extrinsic Dexterity with Parameterized Manipulation Primitives

arXiv: https://arxiv.org/abs/2310.17785 · HTML: https://mlsanity.com/paper/2310.17785/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 26 Oct 2023.

## Builds on (most cited first, 18 of 18)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://mlsanity.com/paper/1710.02298/) (2017)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Automatic Curriculum Learning For Deep RL: A Short Survey](https://mlsanity.com/paper/2003.04664/) (2020)
- [Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning](https://mlsanity.com/paper/1803.09956/) (2018)
- [Learning to Utilize Shaping Rewards: A New Approach of Reward Shaping](https://mlsanity.com/paper/2011.02669/) (2020)
- [Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks](https://mlsanity.com/paper/2110.03655/) (2021)
- [Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives](https://mlsanity.com/paper/2110.15360/) (2021)
- [HACMan: Learning Hybrid Actor-Critic Maps for 6D Non-Prehensile Manipulation](https://mlsanity.com/paper/2305.03942/) (2023)
- [Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity](https://mlsanity.com/paper/2211.01500/) (2022)
- [Efficient learning of goal-oriented push-grasping synergy in clutter](https://mlsanity.com/paper/2103.05405/) (2021)
- [Pre- and post-contact policy decomposition for non-prehensile manipulation with zero-shot sim-to-real transfer](https://mlsanity.com/paper/2309.02754/) (2023)
- [Enhancing Dexterity in Robotic Manipulation via Hierarchical Contact Exploration](https://mlsanity.com/paper/2307.00383/) (2023)
- [Learning Pregrasp Manipulation of Objects from Ungraspable Poses](https://mlsanity.com/paper/2002.06344/) (2020)
- [Learning Generalizable Pivoting Skills](https://mlsanity.com/paper/2305.02554/) (2023)
- [Model Learning for Look-ahead Exploration in Continuous Control](https://mlsanity.com/paper/1811.08086/) (2018)
- [Fast-Learning Grasping and Pre-Grasping via Clutter Quantization and Q-map Masking](https://mlsanity.com/paper/2107.02452/) (2021)

## Built on by (most cited first, 5 of 5)

- [Learning Visuotactile Estimation and Control for Non-prehensile Manipulation under Occlusions](https://mlsanity.com/paper/2412.13157/) (2024)
- [Emerging Extrinsic Dexterity in Cluttered Scenes via Dynamics-aware Policy Learning](https://mlsanity.com/paper/2603.09882/) (2026)
- [On the Fly Adaptation of Behavior Tree-Based Policies through Reinforcement Learning](https://mlsanity.com/paper/2503.06309/) (2025)
- [Leveraging Extrinsic Dexterity for Occluded Grasping on Grasp Constraining Walls](https://mlsanity.com/paper/2507.14721/) (2025)
- [KEA: Keeping Exploration Alive by Proactively Coordinating Exploration Strategies](https://mlsanity.com/paper/2503.18234/) (2025)

## Abstract (arXiv)

Many practically relevant robot grasping problems feature a target object for which all grasps are occluded, e.g., by the environment. Single-shot grasp planning invariably fails in such scenarios. Instead, it is necessary to first manipulate the object into a configuration that affords a grasp. We solve this problem by learning a sequence of actions that utilize the environment to change the object's pose. Concretely, we employ hierarchical reinforcement learning to combine a sequence of learned parameterized manipulation primitives. By learning the low-level manipulation policies, our approach can control the object's state through exploiting interactions between the object, the gripper, and the environment. Designing such a complex behavior analytically would be infeasible under uncontrolled conditions, as an analytic approach requires accurate physical modeling of the interaction and contact dynamics. In contrast, we learn a hierarchical policy model that operates directly on depth perception data, without the need for object detection, pose estimation, or manual design of controllers. We evaluate our approach on picking box-shaped objects of various weight, shape, and friction properties from a constrained table-top workspace. Our method transfers to a real robot and is able to successfully complete the object picking task in 98% of experimental trials. Supplementary information and videos can be found at https://shihminyang.github.io/ED-PMP/.