# Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity

arXiv: https://arxiv.org/abs/2211.01500 · HTML: https://mlsanity.com/paper/2211.01500/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 2 Nov 2022.

## Builds on (most cited first, 28 of 28)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [robosuite: A Modular Simulation Framework and Benchmark for Robot Learning](https://mlsanity.com/paper/2009.12293/) (2020)
- [6-DOF GraspNet: Variational Grasp Generation for Object Manipulation](https://mlsanity.com/paper/1905.10520/) (2019)
- [Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours](https://mlsanity.com/paper/1509.06825/) (2015)
- [Deep Dynamics Models for Learning Dexterous Manipulation](https://mlsanity.com/paper/1909.11652/) (2019)
- [A System for General In-Hand Object Re-Orientation](https://mlsanity.com/paper/2111.03043/) (2021)
- [Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations](https://mlsanity.com/paper/1912.04344/) (2019)
- [Data-Driven Grasp Synthesis - A Survey](https://mlsanity.com/paper/1309.2660/) (2013)
- [Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes](https://mlsanity.com/paper/2110.06192/) (2021)
- [6-DOF Grasping for Target-driven Object Manipulation in Clutter](https://mlsanity.com/paper/1912.03628/) (2019)
- [Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Tasks](https://mlsanity.com/paper/1906.08880/) (2019)
- [Divide-and-Conquer Reinforcement Learning](https://mlsanity.com/paper/1711.09874/) (2017)
- [Same Object, Different Grasps: Data and Semantic Knowledge for Task-Oriented Grasping](https://mlsanity.com/paper/2011.06431/) (2020)
- [A Modular Robotic Arm Control Stack for Research: Franka-Interface and FrankaPy](https://mlsanity.com/paper/2011.02398/) (2020)
- [Goal-Auxiliary Actor-Critic for 6D Robotic Grasping with Point Clouds](https://mlsanity.com/paper/2010.00824/) (2020)
- [Manipulation Trajectory Optimization with Online Grasp Synthesis and Selection](https://mlsanity.com/paper/1911.10280/) (2019)
- [Contact Mode Guided Motion Planning for Quasidynamic Dexterous Manipulation in 3D](https://mlsanity.com/paper/2105.14431/) (2021)
- [Contact Mode Guided Sampling-Based Planning for Quasistatic Dexterous Manipulation in 2D](https://mlsanity.com/paper/2011.01454/) (2020)
- [In-Hand Manipulation via Motion Cones](https://mlsanity.com/paper/1810.00219/) (2018)
- [Learning Pregrasp Manipulation of Objects from Ungraspable Poses](https://mlsanity.com/paper/2002.06344/) (2020)
- [Sampling-based Planning of In-Hand Manipulation with External Pushes](https://mlsanity.com/paper/1707.00318/) (2017)
- [Manipulation with Shared Grasping](https://mlsanity.com/paper/2006.02996/) (2020)

## Built on by (most cited first, 28 of 28)

- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [Robot Parkour Learning](https://mlsanity.com/paper/2309.05665/) (2023)
- [DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Grasping](https://mlsanity.com/paper/2502.20900/) (2025)
- [TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction](https://mlsanity.com/paper/2405.10315/) (2024)
- [HACMan: Learning Hybrid Actor-Critic Maps for 6D Non-Prehensile Manipulation](https://mlsanity.com/paper/2305.03942/) (2023)
- [DyWA: Dynamics-adaptive World Action Model for Generalizable Non-prehensile Manipulation](https://mlsanity.com/paper/2503.16806/) (2025)
- [ArrayBot: Reinforcement Learning for Generalizable Distributed Manipulation through Touch](https://mlsanity.com/paper/2306.16857/) (2023)
- [PreAfford: Universal Affordance-Based Pre-Grasping for Diverse Objects and Environments](https://mlsanity.com/paper/2404.03634/) (2024)
- [RecoveryChaining: Learning Local Recovery Policies for Robust Manipulation](https://mlsanity.com/paper/2410.13979/) (2024)
- [HERD: Continuous Human-to-Robot Evolution for Learning from Human Demonstration](https://mlsanity.com/paper/2212.04359/) (2022)
- [CORN: Contact-based Object Representation for Nonprehensile Manipulation of General Unseen Objects](https://mlsanity.com/paper/2403.10760/) (2024)
- [HACMan++: Spatially-Grounded Motion Primitives for Manipulation](https://mlsanity.com/paper/2407.08585/) (2024)
- [Learning Visuotactile Estimation and Control for Non-prehensile Manipulation under Occlusions](https://mlsanity.com/paper/2412.13157/) (2024)
- [Learning Extrinsic Dexterity with Parameterized Manipulation Primitives](https://mlsanity.com/paper/2310.17785/) (2023)
- [IMPACT: Intelligent Motion Planning with Acceptable Contact Trajectories via Vision-Language Models](https://mlsanity.com/paper/2503.10110/) (2025)
- [Speeding up 6-DoF Grasp Sampling with Quality-Diversity](https://mlsanity.com/paper/2403.06173/) (2024)
- [Dexterous Functional Pre-Grasp Manipulation with Diffusion Policy](https://mlsanity.com/paper/2403.12421/) (2024)
- [Enhancing Exploration with Diffusion Policies in Hybrid Off-Policy RL: Application to Non-Prehensile Manipulation](https://mlsanity.com/paper/2411.14913/) (2024)
- [Collective Intelligence for 2D Push Manipulations with Mobile Robots](https://mlsanity.com/paper/2211.15136/) (2022)
- [Toward a Plug-and-Play Vision-Based Grasping Module for Robotics](https://mlsanity.com/paper/2310.04349/) (2023)
- [QDGset: A Large Scale Grasping Dataset Generated with Quality-Diversity](https://mlsanity.com/paper/2410.02319/) (2024)
- [Learning Long-Horizon Robot Manipulation Skills via Privileged Action](https://mlsanity.com/paper/2502.15442/) (2025)
- [RAPID: Robot Agentic Programming from Demonstrations](https://mlsanity.com/paper/2609.30249/) (2026)
- [COMBO-Grasp: Learning Constraint-Based Manipulation for Bimanual Occluded Grasping](https://mlsanity.com/paper/2502.08054/) (2025)
- [On the Fly Adaptation of Behavior Tree-Based Policies through Reinforcement Learning](https://mlsanity.com/paper/2503.06309/) (2025)
- [Leveraging Extrinsic Dexterity for Occluded Grasping on Grasp Constraining Walls](https://mlsanity.com/paper/2507.14721/) (2025)
- [KEA: Keeping Exploration Alive by Proactively Coordinating Exploration Strategies](https://mlsanity.com/paper/2503.18234/) (2025)
- [Evolving Dexterous Robots from Scratch](https://mlsanity.com/paper/2609.33101/) (2026)

## Abstract (arXiv)

A simple gripper can solve more complex manipulation tasks if it can utilize the external environment such as pushing the object against the table or a vertical wall, known as "Extrinsic Dexterity." Previous work in extrinsic dexterity usually has careful assumptions about contacts which impose restrictions on robot design, robot motions, and the variations of the physical parameters. In this work, we develop a system based on reinforcement learning (RL) to address these limitations. We study the task of "Occluded Grasping" which aims to grasp the object in configurations that are initially occluded; the robot needs to move the object into a configuration from which these grasps can be achieved. We present a system with model-free RL that successfully achieves this task using a simple gripper with extrinsic dexterity. The policy learns emergent behaviors of pushing the object against the wall to rotate and then grasp it without additional reward terms on extrinsic dexterity. We discuss important components of the system including the design of the RL problem, multi-grasp training and selection, and policy generalization with automatic curriculum. Most importantly, the policy trained in simulation is zero-shot transferred to a physical robot. It demonstrates dynamic and contact-rich motions with a simple gripper that generalizes across objects with various size, density, surface friction, and shape with a 78% success rate. Videos can be found at https://sites.google.com/view/grasp-ungraspable/.