# Sampling-based Exploration for Reinforcement Learning of Dexterous Manipulation

arXiv: https://arxiv.org/abs/2303.03486 · HTML: https://mlsanity.com/paper/2303.03486/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Mar 2023.

## Built on by (most cited first, 24 of 24)

- [DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation](https://mlsanity.com/paper/2403.07788/) (2024)
- [General In-Hand Object Rotation with Vision and Touch](https://mlsanity.com/paper/2309.09979/) (2023)
- [DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Grasping](https://mlsanity.com/paper/2502.20900/) (2025)
- [Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulation](https://mlsanity.com/paper/2309.00987/) (2023)
- [Twisting Lids Off with Two Hands](https://mlsanity.com/paper/2403.02338/) (2024)
- [Lessons from Learning to Spin "Pens"](https://mlsanity.com/paper/2407.18902/) (2024)
- [AnyRotate: Gravity-Invariant In-Hand Object Rotation with Sim-to-Real Touch](https://mlsanity.com/paper/2405.07391/) (2024)
- [Learning In-Hand Translation Using Tactile Skin With Shear and Normal Force Sensing](https://mlsanity.com/paper/2407.07885/) (2024)
- [Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration](https://mlsanity.com/paper/2509.09671/) (2025)
- [Estimator-Coupled Reinforcement Learning for Robust Purely Tactile In-Hand Manipulation](https://mlsanity.com/paper/2311.04060/) (2023)
- [From Simple to Complex Skills: The Case of In-Hand Object Reorientation](https://mlsanity.com/paper/2501.05439/) (2025)
- [A roadmap for AI in robotics](https://mlsanity.com/paper/2507.19975/) (2025)
- [R×R: Rapid eXploration for Reinforcement Learning via Sampling-based Reset Distributions and Imitation Pre-training](https://mlsanity.com/paper/2401.15484/) (2024)
- [Composing Dextrous Grasping and In-hand Manipulation via Scoring with a Reinforcement Learning Critic](https://mlsanity.com/paper/2505.13253/) (2025)
- [Dexterous Manipulation Based on Prior Dexterous Grasp Pose Knowledge](https://mlsanity.com/paper/2412.15587/) (2024)
- [Distilling Realizable Students from Unrealizable Teachers](https://mlsanity.com/paper/2505.09546/) (2025)
- [Lightning Grasp: High Performance Procedural Grasp Synthesis with Contact Fields](https://mlsanity.com/paper/2511.07418/) (2025)
- [Real-World Reinforcement Learning with MPC Scaffolding for Dexterous Manipulation](https://mlsanity.com/paper/2609.14878/) (2026)
- [AsymDex: Asymmetry and Relative Coordinates for RL-based Bimanual Dexterity](https://mlsanity.com/paper/2411.13020/) (2024)
- [Multi-Goal Dexterous Hand Manipulation using Probabilistic Model-based Reinforcement Learning](https://mlsanity.com/paper/2504.21585/) (2025)
- [ContactExplorer: Contact Coverage-Guided Exploration for General-Purpose Dexterous Manipulation](https://mlsanity.com/paper/2603.10971/) (2026)
- [Mana: Dexterous Manipulation of Articulated Tools](https://mlsanity.com/paper/2606.13677/) (2026)
- [AURORA: Active Uncertainty-Driven Re-Orientation for In-Hand Reconstruction](https://mlsanity.com/paper/2609.08493/) (2026)
- [Grasp2Twist: Learning Bimanual Dexterous Jar Opening by Reinforcement Learning](https://mlsanity.com/paper/2609.32064/) (2026)

## Abstract (arXiv)

In this paper, we present a novel method for achieving dexterous manipulation of complex objects, while simultaneously securing the object without the use of passive support surfaces. We posit that a key difficulty for training such policies in a Reinforcement Learning framework is the difficulty of exploring the problem state space, as the accessible regions of this space form a complex structure along manifolds of a high-dimensional space. To address this challenge, we use two versions of the non-holonomic Rapidly-Exploring Random Trees algorithm; one version is more general, but requires explicit use of the environment's transition function, while the second version uses manipulation-specific kinematic constraints to attain better sample efficiency. In both cases, we use states found via sampling-based exploration to generate reset distributions that enable training control policies under full dynamic constraints via model-free Reinforcement Learning. We show that these policies are effective at manipulation problems of higher difficulty than previously shown, and also transfer effectively to real robots. Videos of the real-hand demonstrations can be found on the project website: https://sbrl.cs.columbia.edu/