# Composing Dextrous Grasping and In-hand Manipulation via Scoring with a Reinforcement Learning Critic

arXiv: https://arxiv.org/abs/2505.13253 · HTML: https://mlsanity.com/paper/2505.13253/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 24 of its 55 references that are in the map.
- First posted on arXiv 19 May 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [A System for General In-Hand Object Re-Orientation](https://mlsanity.com/paper/2111.03043/) (2021)
- [DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality](https://mlsanity.com/paper/2210.13702/) (2022)
- [In-Hand Object Rotation via Rapid Motor Adaptation](https://mlsanity.com/paper/2210.04887/) (2022)
- [Deep Learning for Detecting Robotic Grasps](https://mlsanity.com/paper/1301.3592/) (2013)
- [Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes](https://mlsanity.com/paper/2211.11744/) (2022)
- [General In-Hand Object Rotation with Vision and Touch](https://mlsanity.com/paper/2309.09979/) (2023)
- [Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing](https://mlsanity.com/paper/2312.01853/) (2023)
- [Complex In-Hand Manipulation via Compliance-Enabled Finger Gaiting and Multi-Modal Planning](https://mlsanity.com/paper/2201.07928/) (2022)
- [Lessons from Learning to Spin "Pens"](https://mlsanity.com/paper/2407.18902/) (2024)
- [Sampling-based Exploration for Reinforcement Learning of Dexterous Manipulation](https://mlsanity.com/paper/2303.03486/) (2023)
- [Learning Purely Tactile In-Hand Manipulation with a Torque-Controlled Hand](https://mlsanity.com/paper/2204.03698/) (2022)
- [Dextrous Tactile In-Hand Manipulation Using a Modular Reinforcement Learning Architecture](https://mlsanity.com/paper/2303.04705/) (2023)
- [DVGG: Deep Variational Grasp Generation for Dextrous Manipulation](https://mlsanity.com/paper/2211.11154/) (2022)
- [Estimator-Coupled Reinforcement Learning for Robust Purely Tactile In-Hand Manipulation](https://mlsanity.com/paper/2311.04060/) (2023)
- [Learning a Shape-Conditioned Agent for Purely Tactile In-Hand Manipulation of Various Objects](https://mlsanity.com/paper/2407.18834/) (2024)
- [Combining Shape Completion and Grasp Prediction for Fast and Versatile Grasping with a Multi-Fingered Hand](https://mlsanity.com/paper/2310.20350/) (2023)
- [R×R: Rapid eXploration for Reinforcement Learning via Sampling-based Reset Distributions and Imitation Pre-training](https://mlsanity.com/paper/2401.15484/) (2024)
- [Self-Contained and Automatic Calibration of a Multi-Fingered Hand Using Only Pairwise Contact Measurements](https://mlsanity.com/paper/2311.03957/) (2023)
- [Self-Contained Calibration of an Elastic Humanoid Upper Body Using Only a Head-Mounted RGB Camera](https://mlsanity.com/paper/2311.08338/) (2023)
- [Speeding Up Optimization-based Motion Planning through Deep Learning](https://mlsanity.com/paper/2311.08345/) (2023)

## Built on by (most cited first, 3 of 3)

- [DexNDM: Closing the Reality Gap for Dexterous In-Hand Rotation via Joint-Wise Neural Dynamics Model](https://mlsanity.com/paper/2510.08556/) (2025)
- [LeLaR: The First In-Orbit Demonstration of an AI-Based Satellite Attitude Controller](https://mlsanity.com/paper/2512.19576/) (2025)
- [Mana: Dexterous Manipulation of Articulated Tools](https://mlsanity.com/paper/2606.13677/) (2026)

## Abstract (arXiv)

In-hand manipulation and grasping are fundamental yet often separately addressed tasks in robotics. For deriving in-hand manipulation policies, reinforcement learning has recently shown great success. However, the derived controllers are not yet useful in real-world scenarios because they often require a human operator to place the objects in suitable initial (grasping) states. Finding stable grasps that also promote the desired in-hand manipulation goal is an open problem. In this work, we propose a method for bridging this gap by leveraging the critic network of a reinforcement learning agent trained for in-hand manipulation to score and select initial grasps. Our experiments show that this method significantly increases the success rate of in-hand manipulation without requiring additional training. We also present an implementation of a full grasp manipulation pipeline on a real-world system, enabling autonomous grasping and reorientation even of unwieldy objects.