# VTLoc: Learning-based Tactile Contact Localization in Visual Point Clouds

arXiv: https://arxiv.org/abs/2607.16146 · HTML: https://mlsanity.com/paper/2607.16146/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 30 references that are in the map.
- First posted on arXiv 17 Jul 2026.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objects](https://mlsanity.com/paper/2306.00956/) (2023)
- [MidasTouch: Monte-Carlo inference over distributions across sliding touch](https://mlsanity.com/paper/2210.14210/) (2022)
- [ViTac: Feature Sharing between Vision and Tactile Sensing for Cloth Texture Recognition](https://mlsanity.com/paper/1802.07490/) (2018)
- [Robotic Tactile Perception of Object Properties: A Review](https://mlsanity.com/paper/1711.03810/) (2017)
- [Tactile Object Pose Estimation from the First Touch with Geometric Contact Rendering](https://mlsanity.com/paper/2012.05205/) (2020)
- [Tactile Mapping and Localization from High-Resolution Tactile Imprints](https://mlsanity.com/paper/1904.10944/) (2019)
- [Tac2Pose: Tactile Object Pose Estimation from the First Touch](https://mlsanity.com/paper/2204.11701/) (2022)
- [S3M-Net: Joint Learning of Semantic Segmentation and Stereo Matching for Autonomous Driving](https://mlsanity.com/paper/2401.11414/) (2024)
- [VIRDO++: Real-World, Visuo-tactile Dynamics and Perception of Deformable Objects](https://mlsanity.com/paper/2210.03701/) (2022)
- [Localizing the Object Contact through Matching Tactile Features with Visual Map](https://mlsanity.com/paper/1708.04441/) (2017)
- [CEDex: Cross-Embodiment Dexterous Grasp Generation at Scale from Human-like Contact Representations](https://mlsanity.com/paper/2509.24661/) (2025)
- [ConViTac: Aligning Visual-Tactile Fusion with Contrastive Representations](https://mlsanity.com/paper/2506.20757/) (2025)
- [HyperTaxel: Hyper-Resolution for Taxel-Based Tactile Signals Through Contrastive Learning](https://mlsanity.com/paper/2408.08312/) (2024)
- [GelSLAM: A Real-time, High-Fidelity, and Robust 3D Tactile SLAM System](https://mlsanity.com/paper/2508.15990/) (2025)
- [ViTacGen: Robotic Pushing with Vision-to-Touch Generation](https://mlsanity.com/paper/2510.14117/) (2025)
- [Training Tactile Sensors to Learn Force Sensing from Each Other](https://mlsanity.com/paper/2503.01058/) (2025)

## Abstract (arXiv)

Vision and touch are complementary modalities essential for robotic perception and manipulation. While vision provides global object context, touch offers precise local information at contact points. Integrating these modalities for contact localization, i.e., predicting the location of touch on an object's surface, poses significant challenges due to the need for accurate spatial alignment between tactile data and visual geometry. To address this challenge, we propose VTLoc, a novel visual-tactile framework that localizes contact points from tactile readings using a 3D point cloud as visual input. VTLoc introduces two key components: a geometric multi-modal alignment module, which reconstructs a pseudo-point cloud from fused visual-tactile features and aligns it with the visual point cloud to enforce spatial consistencies across modalities; and an iterative localizing updater, which iteratively refines the predicted contact location using fused visual-tactile features. Evaluated on a new benchmark of 100 real-world objects, VTLoc improves single-touch contact localization by reducing local-to-global correspondence ambiguity.