mlsanityData to 7 Oct 2026

Rotating without Seeing: Towards In-hand Dexterity through Touch

arXiv 2303.10880 ↗First posted 20 Mar 2023cs.RO cs.AI cs.LGcited by 78

Not on the map. Why?

Abstract (arXiv)

Tactile information plays a critical role in human dexterity. It reveals useful contact information that may not be inferred directly from vision. In fact, humans can even perform in-hand dexterous manipulation without using vision. Can we enable the same ability for the multi-finger robot hand? In this paper, we present Touch Dexterity, a new system that can perform in-hand object rotation using only touching without seeing the object. Instead of relying on precise tactile sensing in a small region, we introduce a new system design using dense binary force sensors (touch or no touch) overlaying one side of the whole robot hand (palm, finger links, fingertips). Such a design is low-cost, giving a larger coverage of the object, and minimizing the Sim2Real gap at the same time. We train an in-hand rotation policy using Reinforcement Learning on diverse objects in simulation. Relying on touch-only sensing, we can directly deploy the policy in a real robot hand and rotate novel objects that are not presented in training. Extensive ablations are performed on how tactile information help in-hand manipulation.Our project is available at https://touchdexterity.github.io.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Builds on 46 of its 75 references that are in the map. First posted on arXiv 20 Mar 2023.

Builds on

46 of 75 references.

References · 46

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  4. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  5. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  6. 2018Learning Dexterous In-Hand Manipulationcited by 643
  7. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 194
  8. 2020DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulationcited by 164
  9. 2021A System for General In-Hand Object Re-Orientationcited by 155
  10. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 136
  11. 2022In-Hand Object Rotation via Rapid Motor Adaptationcited by 102
  12. 2022Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapescited by 92
34 more
  1. 2019Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Taskscited by 80
  2. 2022From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperationcited by 79
  3. 2022DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulationcited by 78
  4. 2022Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulationcited by 75
  5. 2022ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfercited by 73
  6. 2022Holo-Dex: Teaching Dexterity with Immersive Mixed Realitycited by 52
  7. 2022Learning Generalizable Dexterous Manipulation from Human Grasp Affordancecited by 46
  8. 2021Tactile-RL for Insertion: Generalization to Objects of Unknown Geometrycited by 45
  9. 2021Taxim: An Example-based Simulation Model for GelSight Tactile Sensorscited by 42
  10. 2021GelSlim3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Fingercited by 39
  11. 2019Manipulation by Feel: Touch-Based Control with Deep Predictive Modelscited by 34
  12. 2021Active 3D Shape Reconstruction from Vision and Touchcited by 34
  13. 2022Learning Continuous Grasping Function with a Dexterous Hand from Human Demonstrationscited by 34
  14. 2021ReSkin: versatile, replaceable, lasting tactile skinscited by 33
  15. 2022MidasTouch: Monte-Carlo inference over distributions across sliding touchcited by 33
  16. 2022Learning to Imitate Object Interactions from Internet Videoscited by 32
  17. 2020OmniTact: A Multi-Directional High Resolution Touch Sensorcited by 27
  18. 2020Tactile Dexterity: Manipulation Primitives with Tactile Feedbackcited by 21
  19. 2022Complex In-Hand Manipulation via Compliance-Enabled Finger Gaiting and Multi-Modal Planningcited by 21
  20. 2018Learning to Grasp Without Seeingcited by 19
  21. 2021Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translationcited by 17
  22. 2022Learning Purely Tactile In-Hand Manipulation with a Torque-Controlled Handcited by 17
  23. 2023Dextrous Tactile In-Hand Manipulation Using a Modular Reinforcement Learning Architecturecited by 17
  24. 2022Surprisingly Robust In-Hand Manipulation: An Empirical Studycited by 16
  25. 2020Tactile SLAM: Real-time inference of shape and pose from planar pushingcited by 15
  26. 2021Zero-shot sim-to-real transfer of tactile control policies for aggressive swing-up manipulationcited by 13
  27. 2022TANDEM: Learning Joint Exploration and Decision Making with Tactile Sensorscited by 12
  28. 2021PatchGraph: In-hand tactile tracking with learned surface normalscited by 10
  29. 2021On the Feasibility of Learning Finger-gaiting In-hand Manipulation with Intrinsic Sensingcited by 9
  30. 2022HERD: Continuous Human-to-Robot Evolution for Learning from Human Demonstrationcited by 8
  31. 2017Sampling-based Planning of In-Hand Manipulation with External Pushescited by 7
  32. 2021Improving Grasp Stability with Rotation Measurement from Tactile Sensingcited by 6
  33. 2022Learning to Synthesize Volumetric Meshes from Vision-based Tactile Imprintscited by 3
  34. 2022Manipulation of unknown objects via contact configuration regulationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 25; 2025: 19; 2026: 18.

78 papers.

Sort:

By area:

  1. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  2. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 149
  3. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 115
  4. 2024Binding Touch to Everything: Learning Unified Multimodal Tactile Representationscited by 94
  5. 2023General In-Hand Object Rotation with Vision and Touchcited by 87
  6. 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 87
  7. 20243D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensingcited by 67
  8. 2024SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splattingcited by 63
  9. 2023Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Playcited by 60
  10. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 56
  11. 2023Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensingcited by 51
  12. 2023LEMON: Learning 3D Human-Object Interaction Relation from 2D Imagescited by 49
38 more
  1. 2023Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulationcited by 43
  2. 2023Dexterous Functional Graspingcited by 36
  3. 2024Twisting Lids Off with Two Handscited by 36
  4. 2023Dynamic Handover: Throw and Catch with Bimanual Handscited by 33
  5. 2023See to Touch: Learning Tactile Dexterity through Visual Incentivescited by 33
  6. 2024ViViDex: Learning Vision-based Dexterous Manipulation from Human Videoscited by 33
  7. 2023MimicTouch: Leveraging Multi-modal Human Tactile Demonstrations for Contact-rich Manipulationcited by 30
  8. 2025DexMachina: Functional Retargeting for Bimanual Dexterous Manipulationcited by 30
  9. 2025Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Grippercited by 29
  10. 2023Neural feels with neural fields: Visuo-tactile perception for in-hand manipulationcited by 28
  11. 2024Tactile-Augmented Radiance Fieldscited by 26
  12. 2024CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulationcited by 25
  13. 2024Bridging the Human to Robot Dexterity Gap through Object-Oriented Rewardscited by 25
  14. 2023The Power of the Senses: Generalizable Manipulation from Vision and Touch through Masked Multimodal Learningcited by 24
  15. 2024Lessons from Learning to Spin "Pens"cited by 22
  16. 2026LLM-as-a-Verifier: A General-Purpose Verification Frameworkcited by 21
  17. 2023Grasp Multiple Objects with One Handcited by 18
  18. 2024AnyRotate: Gravity-Invariant In-Hand Object Rotation with Sim-to-Real Touchcited by 18
  19. 2025Dexterous Manipulation through Imitation Learning: A Surveycited by 18
  20. 2025CordViP: Correspondence-based Visuomotor Policy for Dexterous Manipulation in Real-Worldcited by 16
  21. 2024Learning In-Hand Translation Using Tactile Skin With Shear and Normal Force Sensingcited by 13
  22. 2024Learning Diverse Bimanual Dexterous Manipulation Skills from Human Demonstrationscited by 12
  23. 2024DexTouch: Learning to Seek and Manipulate Objects with Tactile Dexteritycited by 11
  24. 2023Learning Explicit Contact for Implicit Reconstruction of Hand-held Objects from Monocular Imagescited by 10
  25. 2025VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuningcited by 10
  26. 2023ArrayBot: Reinforcement Learning for Generalizable Distributed Manipulation through Touchcited by 9
  27. 2024Dynamic Reconstruction of Hand-Object Interaction with Distributed Force-aware Contact Representationcited by 8
  28. 2025From Simple to Complex Skills: The Case of In-Hand Object Reorientationcited by 8
  29. 2023Estimator-Coupled Reinforcement Learning for Robust Purely Tactile In-Hand Manipulationcited by 7
  30. 2024Vegetable Peeling: A Case Study in Constrained Dexterous Manipulationcited by 7
  31. 2024DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robotscited by 7
  32. 2024Automatic Environment Shaping is the Next Frontier in RLcited by 6
  33. 2024HyperTaxel: Hyper-Resolution for Taxel-Based Tactile Signals Through Contrastive Learningcited by 6
  34. 2025Enhancing Tactile-based Reinforcement Learning for Robotic Controlcited by 5
  35. 2023In-Hand 3D Object Reconstruction from a Monocular RGB Videocited by 4
  36. 2025A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewardscited by 4
  37. 2025V-HOP: Visuo-Haptic 6D Object Pose Trackingcited by 4
  38. 2026FlexiTac: A Low-Cost, Open-Source, Scalable Tactile Sensing Solution for Robotic Systemscited by 4

50 most cited of 78 All 78 (JSON)