# RetrDex: Efficient Object Retrieval in Cluttered Scenes with a Dexterous Hand

arXiv: https://arxiv.org/abs/2502.18423 · HTML: https://mlsanity.com/paper/2502.18423/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2025.

## Built on by (most cited first, 4 of 4)

- [TransNormal: Dense Visual Semantics for Diffusion-based Transparent Object Normal Estimation](https://mlsanity.com/paper/2602.00839/) (2026)
- [Deformable Cluster Manipulation via Whole-Arm Policy Learning](https://mlsanity.com/paper/2507.17085/) (2025)
- [IPD: Boosting Sequential Policy with Imaginary Planning Distillation in Offline Reinforcement Learning](https://mlsanity.com/paper/2603.04289/) (2026)
- [CABTO: Context-Aware Behavior Tree Grounding for Robot Manipulation](https://mlsanity.com/paper/2603.16809/) (2026)

## Abstract (arXiv)

Retrieving objects buried beneath clutter is both challenging and time-consuming, as complex support relationships make manipulation particularly difficult. Existing methods either focus on support relations and rely on sequential grasping to remove occluding objects, or perform preparatory actions such as pushing to facilitate subsequent grasps. However, these approaches are often inefficient and treat physical interactions as isolated auxiliary steps. In this paper, we propose RetrDex, an efficient framework for dexterous arm-hand systems to learn object retrieval in cluttered scenes. Our approach leverages large-scale parallel reinforcement learning (RL) in diverse cluttered scenes and incorporates a spatially aware representation that encodes occlusion patterns and spatial relationships among the target, the dexterous hand, and surrounding clutter. This representation enables the policy to develop diverse manipulation skills (e.g., pushing, stirring, and poking) that actively clear occluders. We evaluate RetrDex on 16 household objects across varied clutter configurations, and obtain strong retrieval performance and efficiency on both seen and unseen targets. Furthermore, we demonstrate successful zero-shot transfer to a real-world dexterous multi-fingered robot system, validating the practical applicability of our method. Videos can be found on our project website: https://RetrDex.github.io.