# In-Hand Object Rotation via Rapid Motor Adaptation

arXiv: https://arxiv.org/abs/2210.04887 · HTML: https://mlsanity.com/paper/2210.04887/

## Key facts

- Cited by 118 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 10 Oct 2022.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Deep Dynamics Models for Learning Dexterous Manipulation](https://mlsanity.com/paper/1909.11652/) (2019)
- [A System for General In-Hand Object Re-Orientation](https://mlsanity.com/paper/2111.03043/) (2021)
- [State-Only Imitation Learning for Dexterous Manipulation](https://mlsanity.com/paper/2004.04650/) (2020)
- [Coupling Vision and Proprioception for Navigation of Legged Robots](https://mlsanity.com/paper/2112.02094/) (2021)
- [Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning](https://mlsanity.com/paper/2111.03062/) (2021)
- [Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger](https://mlsanity.com/paper/2108.09779/) (2021)
- [Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstration](https://mlsanity.com/paper/1603.06348/) (2016)
- [Learning Dexterous Manipulation Policies from Experience and Imitation](https://mlsanity.com/paper/1611.05095/) (2016)
- [Complex In-Hand Manipulation via Compliance-Enabled Finger Gaiting and Multi-Modal Planning](https://mlsanity.com/paper/2201.07928/) (2022)
- [Learning Purely Tactile In-Hand Manipulation with a Torque-Controlled Hand](https://mlsanity.com/paper/2204.03698/) (2022)
- [Relaxed-Rigidity Constraints: Kinematic Trajectory Optimization and Collision Avoidance for In-Grasp Manipulation](https://mlsanity.com/paper/1806.00942/) (2018)
- [On the Feasibility of Learning Finger-gaiting In-hand Manipulation with Intrinsic Sensing](https://mlsanity.com/paper/2109.12720/) (2021)

## Built on by (most cited first, 30 of 118)

- [FAST: Efficient Action Tokenization for Vision-Language-Action Models](https://mlsanity.com/paper/2501.09747/) (2025)
- [Evaluating Real-World Robot Manipulation Policies in Simulation](https://mlsanity.com/paper/2405.05941/) (2024)
- [DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation](https://mlsanity.com/paper/2403.07788/) (2024)
- [Legged Locomotion in Challenging Terrains using Egocentric Vision](https://mlsanity.com/paper/2211.07638/) (2022)
- [ShapeLLM: Universal 3D Object Understanding for Embodied Interaction](https://mlsanity.com/paper/2402.17766/) (2024)
- [ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills](https://mlsanity.com/paper/2502.01143/) (2025)
- [3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations](https://mlsanity.com/paper/2403.03954/) (2024)
- [UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learning](https://mlsanity.com/paper/2304.00464/) (2023)
- [General In-Hand Object Rotation with Vision and Touch](https://mlsanity.com/paper/2309.09979/) (2023)
- [DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Grasping](https://mlsanity.com/paper/2502.20900/) (2025)
- [Rotating without Seeing: Towards In-hand Dexterity through Touch](https://mlsanity.com/paper/2303.10880/) (2023)
- [DrEureka: Language Model Guided Sim-To-Real Transfer](https://mlsanity.com/paper/2406.01967/) (2024)
- [LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning](https://mlsanity.com/paper/2309.06440/) (2023)
- [Learning Visuotactile Skills with Two Multifingered Hands](https://mlsanity.com/paper/2404.16823/) (2024)
- [SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting](https://mlsanity.com/paper/2409.10161/) (2024)
- [TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction](https://mlsanity.com/paper/2405.10315/) (2024)
- [Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing](https://mlsanity.com/paper/2312.01853/) (2023)
- [RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning](https://mlsanity.com/paper/2504.18904/) (2025)
- [Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulation](https://mlsanity.com/paper/2309.00987/) (2023)
- [Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids](https://mlsanity.com/paper/2502.20396/) (2025)
- [Twisting Lids Off with Two Hands](https://mlsanity.com/paper/2403.02338/) (2024)
- [Dexterous Functional Grasping](https://mlsanity.com/paper/2312.02975/) (2023)
- [Hand-Object Interaction Pretraining from Videos](https://mlsanity.com/paper/2409.08273/) (2024)
- [DexMachina: Functional Retargeting for Bimanual Dexterous Manipulation](https://mlsanity.com/paper/2505.24853/) (2025)
- [Dynamic Handover: Throw and Catch with Bimanual Hands](https://mlsanity.com/paper/2309.05655/) (2023)
- [ViViDex: Learning Vision-based Dexterous Manipulation from Human Videos](https://mlsanity.com/paper/2404.15709/) (2024)
- [Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstration](https://mlsanity.com/paper/2504.12609/) (2025)
- [World Models for Learning Dexterous Hand-Object Interactions from Human Videos](https://mlsanity.com/paper/2512.13644/) (2025)
- [RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.04150/) (2023)
- [Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation](https://mlsanity.com/paper/2312.13469/) (2023)

## Abstract (arXiv)

Generalized in-hand manipulation has long been an unsolved challenge of robotics. As a small step towards this grand goal, we demonstrate how to design and learn a simple adaptive controller to achieve in-hand object rotation using only fingertips. The controller is trained entirely in simulation on only cylindrical objects, which then - without any fine-tuning - can be directly deployed to a real robot hand to rotate dozens of objects with diverse sizes, shapes, and weights over the z-axis. This is achieved via rapid online adaptation of the controller to the object properties using only proprioception history. Furthermore, natural and stable finger gaits automatically emerge from training the control policy via reinforcement learning. Code and more videos are available at https://haozhi.io/hora