# Rotating without Seeing: Towards In-hand Dexterity through Touch

arXiv: https://arxiv.org/abs/2303.10880 · HTML: https://mlsanity.com/paper/2303.10880/

## Key facts

- Cited by 84 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 75 references that are in the map.
- First posted on arXiv 20 Mar 2023.

## Builds on (most cited first, 30 of 46)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [DexMV: Imitation Learning for Dexterous Manipulation from Human Videos](https://mlsanity.com/paper/2108.05877/) (2021)
- [DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation](https://mlsanity.com/paper/2005.14679/) (2020)
- [A System for General In-Hand Object Re-Orientation](https://mlsanity.com/paper/2111.03043/) (2021)
- [DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality](https://mlsanity.com/paper/2210.13702/) (2022)
- [In-Hand Object Rotation via Rapid Motor Adaptation](https://mlsanity.com/paper/2210.04887/) (2022)
- [Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes](https://mlsanity.com/paper/2211.11744/) (2022)
- [Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks](https://mlsanity.com/paper/1907.13098/) (2019)
- [DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation](https://mlsanity.com/paper/2211.09423/) (2022)
- [From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperation](https://mlsanity.com/paper/2204.12490/) (2022)
- [Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation](https://mlsanity.com/paper/2203.13251/) (2022)
- [ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer](https://mlsanity.com/paper/2204.02389/) (2022)
- [Holo-Dex: Teaching Dexterity with Immersive Mixed Reality](https://mlsanity.com/paper/2210.06463/) (2022)
- [Learning Generalizable Dexterous Manipulation from Human Grasp Affordance](https://mlsanity.com/paper/2204.02320/) (2022)
- [Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry](https://mlsanity.com/paper/2104.01167/) (2021)
- [Taxim: An Example-based Simulation Model for GelSight Tactile Sensors](https://mlsanity.com/paper/2109.04027/) (2021)
- [GelSlim3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Finger](https://mlsanity.com/paper/2103.12269/) (2021)
- [Manipulation by Feel: Touch-Based Control with Deep Predictive Models](https://mlsanity.com/paper/1903.04128/) (2019)
- [ReSkin: versatile, replaceable, lasting tactile skins](https://mlsanity.com/paper/2111.00071/) (2021)
- [Active 3D Shape Reconstruction from Vision and Touch](https://mlsanity.com/paper/2107.09584/) (2021)
- [Learning Continuous Grasping Function with a Dexterous Hand from Human Demonstrations](https://mlsanity.com/paper/2207.05053/) (2022)
- [Learning to Imitate Object Interactions from Internet Videos](https://mlsanity.com/paper/2211.13225/) (2022)
- [MidasTouch: Monte-Carlo inference over distributions across sliding touch](https://mlsanity.com/paper/2210.14210/) (2022)
- [OmniTact: A Multi-Directional High Resolution Touch Sensor](https://mlsanity.com/paper/2003.06965/) (2020)
- [Complex In-Hand Manipulation via Compliance-Enabled Finger Gaiting and Multi-Modal Planning](https://mlsanity.com/paper/2201.07928/) (2022)

## Built on by (most cited first, 30 of 84)

- [Evaluating Real-World Robot Manipulation Policies in Simulation](https://mlsanity.com/paper/2405.05941/) (2024)
- [DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation](https://mlsanity.com/paper/2403.07788/) (2024)
- [3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations](https://mlsanity.com/paper/2403.03954/) (2024)
- [General In-Hand Object Rotation with Vision and Touch](https://mlsanity.com/paper/2309.09979/) (2023)
- [Binding Touch to Everything: Learning Unified Multimodal Tactile Representations](https://mlsanity.com/paper/2401.18084/) (2024)
- [DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Grasping](https://mlsanity.com/paper/2502.20900/) (2025)
- [3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing](https://mlsanity.com/paper/2410.24091/) (2024)
- [SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting](https://mlsanity.com/paper/2409.10161/) (2024)
- [ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning](https://mlsanity.com/paper/2503.21860/) (2025)
- [Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play](https://mlsanity.com/paper/2303.12076/) (2023)
- [Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing](https://mlsanity.com/paper/2312.01853/) (2023)
- [Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulation](https://mlsanity.com/paper/2309.00987/) (2023)
- [LEMON: Learning 3D Human-Object Interaction Relation from 2D Images](https://mlsanity.com/paper/2312.08963/) (2023)
- [Twisting Lids Off with Two Hands](https://mlsanity.com/paper/2403.02338/) (2024)
- [Dexterous Functional Grasping](https://mlsanity.com/paper/2312.02975/) (2023)
- [DexMachina: Functional Retargeting for Bimanual Dexterous Manipulation](https://mlsanity.com/paper/2505.24853/) (2025)
- [Dynamic Handover: Throw and Catch with Bimanual Hands](https://mlsanity.com/paper/2309.05655/) (2023)
- [See to Touch: Learning Tactile Dexterity through Visual Incentives](https://mlsanity.com/paper/2309.12300/) (2023)
- [ViViDex: Learning Vision-based Dexterous Manipulation from Human Videos](https://mlsanity.com/paper/2404.15709/) (2024)
- [MimicTouch: Leveraging Multi-modal Human Tactile Demonstrations for Contact-rich Manipulation](https://mlsanity.com/paper/2310.16917/) (2023)
- [LLM-as-a-Verifier: A General-Purpose Verification Framework](https://mlsanity.com/paper/2607.05391/) (2026)
- [Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper](https://mlsanity.com/paper/2507.15062/) (2025)
- [Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation](https://mlsanity.com/paper/2312.13469/) (2023)
- [CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulation](https://mlsanity.com/paper/2402.14795/) (2024)
- [Bridging the Human to Robot Dexterity Gap through Object-Oriented Rewards](https://mlsanity.com/paper/2410.23289/) (2024)
- [The Power of the Senses: Generalizable Manipulation from Vision and Touch through Masked Multimodal Learning](https://mlsanity.com/paper/2311.00924/) (2023)
- [Tactile-Augmented Radiance Fields](https://mlsanity.com/paper/2405.04534/) (2024)
- [Lessons from Learning to Spin "Pens"](https://mlsanity.com/paper/2407.18902/) (2024)
- [AnyRotate: Gravity-Invariant In-Hand Object Rotation with Sim-to-Real Touch](https://mlsanity.com/paper/2405.07391/) (2024)
- [Dexterous Manipulation through Imitation Learning: A Survey](https://mlsanity.com/paper/2504.03515/) (2025)

## Abstract (arXiv)

Tactile information plays a critical role in human dexterity. It reveals useful contact information that may not be inferred directly from vision. In fact, humans can even perform in-hand dexterous manipulation without using vision. Can we enable the same ability for the multi-finger robot hand? In this paper, we present Touch Dexterity, a new system that can perform in-hand object rotation using only touching without seeing the object. Instead of relying on precise tactile sensing in a small region, we introduce a new system design using dense binary force sensors (touch or no touch) overlaying one side of the whole robot hand (palm, finger links, fingertips). Such a design is low-cost, giving a larger coverage of the object, and minimizing the Sim2Real gap at the same time. We train an in-hand rotation policy using Reinforcement Learning on diverse objects in simulation. Relying on touch-only sensing, we can directly deploy the policy in a real robot hand and rotate novel objects that are not presented in training. Extensive ablations are performed on how tactile information help in-hand manipulation.Our project is available at https://touchdexterity.github.io.