# Tactile Object Pose Estimation from the First Touch with Geometric Contact Rendering

arXiv: https://arxiv.org/abs/2012.05205 · HTML: https://mlsanity.com/paper/2012.05205/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2020.

## Built on by (most cited first, 20 of 20)

- [Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI](https://mlsanity.com/paper/2407.06886/) (2024)
- [ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer](https://mlsanity.com/paper/2204.02389/) (2022)
- [See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation](https://mlsanity.com/paper/2212.03858/) (2022)
- [Factory: Fast Contact for Robotic Assembly](https://mlsanity.com/paper/2205.03532/) (2022)
- [nerf2nerf: Pairwise Registration of Neural Radiance Fields](https://mlsanity.com/paper/2211.01600/) (2022)
- [Learning to Read Braille: Bridging the Tactile Reality Gap with Diffusion Models](https://mlsanity.com/paper/2304.01182/) (2023)
- [Tac2Pose: Tactile Object Pose Estimation from the First Touch](https://mlsanity.com/paper/2204.11701/) (2022)
- [Zero-shot sim-to-real transfer of tactile control policies for aggressive swing-up manipulation](https://mlsanity.com/paper/2101.02680/) (2021)
- [ViHOPE: Visuotactile In-Hand Object 6D Pose Estimation with Shape Completion](https://mlsanity.com/paper/2309.05662/) (2023)
- [Neural Contact Fields: Tracking Extrinsic Contact with Tactile Sensing](https://mlsanity.com/paper/2210.09297/) (2022)
- [Sim2Real Manipulation on Unknown Objects with Tactile-based Reinforcement Learning](https://mlsanity.com/paper/2403.12170/) (2024)
- [Collision-aware In-hand 6D Object Pose Estimation using Multiple Vision-based Tactile Sensors](https://mlsanity.com/paper/2301.13667/) (2023)
- [Hierarchical Graph Neural Networks for Proprioceptive 6D Pose Estimation of In-hand Objects](https://mlsanity.com/paper/2306.15858/) (2023)
- [HyperTaxel: Hyper-Resolution for Taxel-Based Tactile Signals Through Contrastive Learning](https://mlsanity.com/paper/2408.08312/) (2024)
- [Learning to Synthesize Volumetric Meshes from Vision-based Tactile Imprints](https://mlsanity.com/paper/2203.15155/) (2022)
- [ViTa-Zero: Zero-shot Visuotactile Object 6D Pose Estimation](https://mlsanity.com/paper/2504.13179/) (2025)
- [Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learning](https://mlsanity.com/paper/2606.11767/) (2026)
- [Object Pose Estimation through Dexterous Touch](https://mlsanity.com/paper/2509.13591/) (2025)
- [UniTac2Pose: A Unified Approach Learned in Simulation for Category-level Visuotactile In-hand Pose Estimation](https://mlsanity.com/paper/2509.15934/) (2025)
- [VTLoc: Learning-based Tactile Contact Localization in Visual Point Clouds](https://mlsanity.com/paper/2607.16146/) (2026)

## Abstract (arXiv)

In this paper, we present an approach to tactile pose estimation from the first touch for known objects. First, we create an object-agnostic map from real tactile observations to contact shapes. Next, for a new object with known geometry, we learn a tailored perception model completely in simulation. To do so, we simulate the contact shapes that a dense set of object poses would produce on the sensor. Then, given a new contact shape obtained from the sensor output, we match it against the pre-computed set using the object-specific embedding learned purely in simulation using contrastive learning. This results in a perception model that can localize objects from a single tactile observation. It also allows reasoning over pose distributions and including additional pose constraints coming from other perception systems or multiple contacts. We provide quantitative results for four objects. Our approach provides high accuracy pose estimations from distinctive tactile observations while regressing pose distributions to account for those contact shapes that could result from different object poses. We further extend and test our approach in multi-contact scenarios where several tactile sensors are simultaneously in contact with the object. Website: http://mcube.mit.edu/research/tactile_loc_first_touch.html