# Dexterous In-hand Manipulation by Guiding Exploration with Simple Sub-skill Controllers

arXiv: https://arxiv.org/abs/2303.03533 · HTML: https://mlsanity.com/paper/2303.03533/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 6 Mar 2023.

## Builds on (most cited first, 19 of 19)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [A System for General In-Hand Object Re-Orientation](https://mlsanity.com/paper/2111.03043/) (2021)
- [Reinforcement and Imitation Learning via Interactive No-Regret Learning](https://mlsanity.com/paper/1406.5979/) (2014)
- [In-Hand Object Rotation via Rapid Motor Adaptation](https://mlsanity.com/paper/2210.04887/) (2022)
- [State-Only Imitation Learning for Dexterous Manipulation](https://mlsanity.com/paper/2004.04650/) (2020)
- [Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger](https://mlsanity.com/paper/2108.09779/) (2021)
- [AC-Teach: A Bayesian Actor-Critic Method for Policy Learning with an Ensemble of Suboptimal Teachers](https://mlsanity.com/paper/1909.04121/) (2019)
- [Learning Dexterous Manipulation from Suboptimal Experts](https://mlsanity.com/paper/2010.08587/) (2020)
- [On the Feasibility of Learning Finger-gaiting In-hand Manipulation with Intrinsic Sensing](https://mlsanity.com/paper/2109.12720/) (2021)
- [Geometric In-Hand Regrasp Planning: Alternating Optimization of Finger Gaits and In-Grasp Manipulation](https://mlsanity.com/paper/1804.04292/) (2018)
- [Learning Hierarchical Control for Robust In-Hand Manipulation](https://mlsanity.com/paper/1910.10985/) (2019)
- [TrajectoTree: Trajectory Optimization Meets Tree Search for Planning Multi-contact Dexterous Manipulation](https://mlsanity.com/paper/2109.14088/) (2021)
- [Real-Time Robust Finger Gaits Planning under Object Shape and Dynamics Uncertainties](https://mlsanity.com/paper/1710.10350/) (2017)
- [In-Hand Object Stabilization by Independent Finger Control](https://mlsanity.com/paper/1806.05031/) (2018)

## Built on by (most cited first, 4 of 4)

- [DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots](https://mlsanity.com/paper/2409.06613/) (2024)
- [From Simple to Complex Skills: The Case of In-Hand Object Reorientation](https://mlsanity.com/paper/2501.05439/) (2025)
- [Diffusion-Informed Probabilistic Contact Search for Multi-Finger Manipulation](https://mlsanity.com/paper/2410.00841/) (2024)
- [I Can Hear You Coming: RF Sensing for Uncooperative Satellite Evasion](https://mlsanity.com/paper/2504.03983/) (2025)

## Abstract (arXiv)

Recently, reinforcement learning has led to dexterous manipulation skills of increasing complexity. Nonetheless, learning these skills in simulation still exhibits poor sample-efficiency which stems from the fact these skills are learned from scratch without the benefit of any domain expertise. In this work, we aim to improve the sample efficiency of learning dexterous in-hand manipulation skills using controllers available via domain knowledge. To this end, we design simple sub-skill controllers and demonstrate improved sample efficiency using a framework that guides exploration toward relevant state space by following actions from these controllers. We are the first to demonstrate learning hard-to-explore finger-gaiting in-hand manipulation skills without the use of an exploratory reset distribution. Video results can be found at https://roamlab.github.io/vge