mlsanityData to 14 Oct 2026

ViTaSCOPE: Visuo-tactile Implicit Representation for In-hand Pose and Extrinsic Contact Estimation

arXiv 2506.12239 ↗First posted 13 Jun 2025cs.RO cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Mastering dexterous, contact-rich object manipulation demands precise estimation of both in-hand object poses and external contact locationsunicodex2013tasks particularly challenging due to partial and noisy observations. We present ViTaSCOPE: Visuo-Tactile Simultaneous Contact and Object Pose Estimation, an object-centric neural implicit representation that fuses vision and high-resolution tactile feedback. By representing objects as signed distance fields and distributed tactile feedback as neural shear fields, ViTaSCOPE accurately localizes objects and registers extrinsic contacts onto their 3D geometry as contact fields. Our method enables seamless reasoning over complementary visuo-tactile cues by leveraging simulation for scalable training and zero-shot transfers to the real-world by bridging the sim-to-real gap. We evaluate our method through comprehensive simulated and real-world experiments, demonstrating its capabilities in dexterous manipulation scenarios.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 35 of its 46 references that are in the map. First posted on arXiv 13 Jun 2025.

Builds on

35 of 46 references.

References · 35

  1. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  2. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  3. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  4. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 429
  5. 2020DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulationcited by 167
  6. 2022What's in your hands? 3D Reconstruction of Generic Objects in Handscited by 90
  7. 2020TACTO: A Fast, Flexible, and Open-source Simulator for High-Resolution Vision-based Tactile Sensorscited by 63
  8. 2022AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstructioncited by 59
  9. 2021Taxim: An Example-based Simulation Model for GelSight Tactile Sensorscited by 43
  10. 20203D Shape Reconstruction from Vision and Touchcited by 42
  11. 2021GelSlim3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Fingercited by 41
  12. 2021Active 3D Shape Reconstruction from Vision and Touchcited by 34
23 more
  1. 2024TacSL: A Library for Visuotactile Sensor Simulation and Learningcited by 27
  2. 2023The Power of the Senses: Generalizable Manipulation from Vision and Touch through Masked Multimodal Learningcited by 26
  3. 2024DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulationcited by 26
  4. 2022VIRDO: Visio-tactile Implicit Representations of Deformable Objectscited by 25
  5. 2024Digitizing Touch with an Artificial Multimodal Fingertipcited by 20
  6. 2018Dense Tactile Force Distribution Estimation using GelSlim and inverse FEMcited by 18
  7. 2019Soft-bubble: A highly compliant dense geometry tactile sensor for robot manipulationcited by 16
  8. 2022Tac2Pose: Tactile Object Pose Estimation from the First Touchcited by 15
  9. 2023ViHOPE: Visuotactile In-Hand Object 6D Pose Estimation with Shape Completioncited by 14
  10. 2021Extrinsic Contact Sensing with Relative-Motion Tracking from Distributed Tactile Measurementscited by 13
  11. 2022VIRDO++: Real-World, Visuo-tactile Dynamics and Perception of Deformable Objectscited by 13
  12. 2022Neural Contact Fields: Tracking Extrinsic Contact with Tactile Sensingcited by 13
  13. 2019Tactile-Based Insertion for Dense Box-Packingcited by 12
  14. 2022Manipulation via Membranes: High-Resolution and Highly Deformable Tactile Sensing and Controlcited by 6
  15. 2023Simultaneous Tactile Estimation and Control of Extrinsic Contactcited by 6
  16. 2024Tactile-Driven Non-Prehensile Object Manipulation via Extrinsic Contact Mode Controlcited by 6
  17. 2023Perceiving Extrinsic Contacts from Touch Improves Learning Insertion Policiescited by 4
  18. 2024TEXterity: Tactile Extrinsic deXteritycited by 4
  19. 2024GelSlim 4.0: Focusing on Touch and Reproducibilitycited by 4
  20. 2022Simultaneous Contact Location and Object Pose Estimation Using Proprioception and Tactile Feedbackcited by 3
  21. 2023Integrated Object Deformation and Contact Patch Estimation from Visuo-Tactile Feedbackcited by 3
  22. 2024Visual-auditory Extrinsic Contact Estimationcited by 3
  23. 2023MultiSCOPE: Disambiguating In-Hand Object Poses with Proprioception and Tactile Feedbackcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026HydroShear: Hydroelastic Shear Simulation for Tactile Sim-to-Real Reinforcement Learningcited by 4
  2. 2026Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learningcited by 3
  3. 2026UNIC: Learning Unified Multimodal Extrinsic Contact Estimationcited by 0