mlsanityData to 7 Oct 2026

OPENTOUCH: Bringing Full-Hand Touch to Real-World Interaction

arXiv 2512.16842 ↗First posted 18 Dec 2025cs.CV cs.AI cs.ROcited by 11

Not on the map. Why?

Abstract (arXiv)

The human hand is our primary interface to the physical world, yet egocentric perception rarely knows when, where, or how forcefully it makes contact. Robust wearable tactile sensors are scarce, and no existing in-the-wild datasets align first-person video with full-hand touch. To bridge the gap between visual perception and physical interaction, we present OpenTouch, the first in-the-wild egocentric full-hand tactile dataset, containing 5.1 hours of synchronized video-touch-pose data and 2,900 curated clips with detailed text annotations. Using OpenTouch, we introduce retrieval and classification benchmarks that probe how touch grounds perception and action. We show that tactile signals provide a compact yet powerful cue for grasp understanding, strengthen cross-modal alignment, and can be reliably retrieved from in-the-wild video queries. By releasing this annotated vision-touch-pose dataset and benchmark, we aim to advance multimodal egocentric perception, embodied learning, and contact-rich robotic manipulation.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 59 of its 82 references that are in the map. First posted on arXiv 18 Dec 2025.

Builds on

59 of 82 references.

References · 59

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  5. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  6. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  7. 2025DINOv3cited by 1,297
  8. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  9. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  10. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
  11. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  12. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
38 more
  1. 2020Rescaling Egocentric Visioncited by 529
  2. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  3. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
  4. 2020Multi-modal Transformer for Video Retrievalcited by 395
  5. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
  6. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 335
  7. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  8. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 320
  9. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
  10. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  11. 2020ContactPose: A Dataset of Grasps with Object Contact and Hand Posecited by 180
  12. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 160
  13. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 149
  14. 2021EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devicescited by 136
  15. 2022OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interactioncited by 129
  16. 2024HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videoscited by 122
  17. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  18. 2025EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videoscited by 116
  19. 2024Binding Touch to Everything: Learning Unified Multimodal Tactile Representationscited by 94
  20. 2024EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real Worldcited by 93
  21. 2022Touch and Go: Learning from Human-Collected Vision and Touchcited by 84
  22. 2022ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfercited by 73
  23. 2021ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representationscited by 67
  24. 2019Connecting Touch and Vision via Cross-Modal Predictioncited by 66
  25. 2023EgoHumans: An Egocentric 3D Multi-Human Benchmarkcited by 58
  26. 2025DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulationcited by 58
  27. 2018Light Gated Recurrent Units for Speech Recognitioncited by 48
  28. 2023The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objectscited by 40
  29. 2023EgoObjects: A Large-Scale Egocentric Dataset for Fine-Grained Object Understandingcited by 39
  30. 2024EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Video Generationcited by 38
  31. 2024EgoExo-Fitness: Towards Egocentric and Exocentric Full-Body Action Understandingcited by 35
  32. 2024Learning Precise Affordances from Egocentric Videos for Robotic Manipulationcited by 34
  33. 2025Vision in Action: Learning Active Perception from Human Demonstrationscited by 33
  34. 2024GigaHands: A Massive Annotated Dataset of Bimanual Hand Activitiescited by 32
  35. 2025Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Grippercited by 29
  36. 2022PressureVision: Estimating Hand Pressure from a Single RGB Imagecited by 23
  37. 2025EMMA: Scaling Mobile Manipulation via Egocentric Human Datacited by 20
  38. 2024EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Visioncited by 18

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 11.

11 papers.

  1. 2026FEEL (Force-Enhanced Egocentric Learning): A Dataset for Physical Action Understandingcited by 4
  2. 2026Human-Centric Transferable Tactile Pre-Training for Dexterous Robotic Manipulationcited by 4
  3. 2026EgoTactile: Learning Grasp Pressure for Everyday Objects from Egocentric Videocited by 3
  4. 2026TacCoRL: Integrating Tactile Feedback into VLA via Simulationcited by 3
  5. 2026Tactile-based Multimodal Fusion in Embodied Intelligence: A Survey of Vision, Language, and Contact-Driven Paradigmscited by 2
  6. 2026EgoTac: In-the-wild Tactile Prediction from Egocentric Visioncited by 2
  7. 2026TouchSight: Bare-Handed Tactile Prediction from Egocentric Video via Generative Visual Augmentationcited by 1
  8. 2026A Vision for Multisensory Intelligence: Sensing, Science, and Synergycited by 0
  9. 2026DexTouch-WM: Learning Action-Conditioned Tactile World Models from Human Touch for Dexterous Robot Manipulationcited by 0
  10. 2026TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learningcited by 0
  11. 2026TACROSS: An Efficient and Low-Cost Scalable Human Touch System Across Heterogeneous Tactile Sensors for Dexterous Robot Learningcited by 0