# TactoFind: A Tactile Only System for Object Retrieval

arXiv: https://arxiv.org/abs/2303.13482 · HTML: https://mlsanity.com/paper/2303.13482/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 51 references that are in the map.
- First posted on arXiv 23 Mar 2023.

## Builds on (most cited first, 23 of 23)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation](https://mlsanity.com/paper/2005.14679/) (2020)
- [A System for General In-Hand Object Re-Orientation](https://mlsanity.com/paper/2111.03043/) (2021)
- [ROBEL: Robotics Benchmarks for Learning with Low-Cost Robots](https://mlsanity.com/paper/1909.11639/) (2019)
- [Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry](https://mlsanity.com/paper/2104.01167/) (2021)
- [Cable Manipulation with a Tactile-Reactive Gripper](https://mlsanity.com/paper/1910.02860/) (2019)
- [ReSkin: versatile, replaceable, lasting tactile skins](https://mlsanity.com/paper/2111.00071/) (2021)
- [Active Clothing Material Perception using Tactile Sensing and Deep Learning](https://mlsanity.com/paper/1711.00574/) (2017)
- [SwingBot: Learning Physical Features from In-hand Tactile Exploration for Dynamic Swing-up Manipulation](https://mlsanity.com/paper/2101.11812/) (2021)
- [Learning to Grasp Without Seeing](https://mlsanity.com/paper/1805.04201/) (2018)
- [Tactile Mapping and Localization from High-Resolution Tactile Imprints](https://mlsanity.com/paper/1904.10944/) (2019)
- [Soft-Bubble grippers for robust and perceptive manipulation](https://mlsanity.com/paper/2004.03691/) (2020)
- [Learning to Identify Object Instances by Touch: Tactile Recognition via Multimodal Matching](https://mlsanity.com/paper/1903.03591/) (2019)
- [TANDEM: Learning Joint Exploration and Decision Making with Tactile Sensors](https://mlsanity.com/paper/2203.00798/) (2022)
- [Localizing the Object Contact through Matching Tactile Features with Visual Map](https://mlsanity.com/paper/1708.04441/) (2017)
- [A soft thumb-sized vision-based sensor with accurate all-round force perception](https://mlsanity.com/paper/2111.05934/) (2021)
- [Dynamic Modeling of Hand-Object Interactions via Tactile Sensing](https://mlsanity.com/paper/2109.04378/) (2021)
- [Using Collocated Vision and Tactile Sensors for Visual Servoing and Localization](https://mlsanity.com/paper/2204.11686/) (2022)
- [Object recognition for robotics from tactile time series data utilising different neural network architectures](https://mlsanity.com/paper/2109.04573/) (2021)

## Built on by (most cited first, 5 of 5)

- [DexTouch: Learning to Seek and Manipulate Objects with Tactile Dexterity](https://mlsanity.com/paper/2401.12496/) (2024)
- [ArrayBot: Reinforcement Learning for Generalizable Distributed Manipulation through Touch](https://mlsanity.com/paper/2306.16857/) (2023)
- [Gentle Object Retraction in Dense Clutter Using Multimodal Force Sensing and Imitation Learning](https://mlsanity.com/paper/2508.19476/) (2025)
- [Object Pose Estimation through Dexterous Touch](https://mlsanity.com/paper/2509.13591/) (2025)
- [TACTFUL: Tactile-Driven Exploration For Object Localization and Identification in Confined Environments](https://mlsanity.com/paper/2606.24712/) (2026)

## Abstract (arXiv)

We study the problem of object retrieval in scenarios where visual sensing is absent, object shapes are unknown beforehand and objects can move freely, like grabbing objects out of a drawer. Successful solutions require localizing free objects, identifying specific object instances, and then grasping the identified objects, only using touch feedback. Unlike vision, where cameras can observe the entire scene, touch sensors are local and only observe parts of the scene that are in contact with the manipulator. Moreover, information gathering via touch sensors necessitates applying forces on the touched surface which may disturb the scene itself. Reasoning with touch, therefore, requires careful exploration and integration of information over time -- a challenge we tackle. We present a system capable of using sparse tactile feedback from fingertip touch sensors on a dexterous hand to localize, identify and grasp novel objects without any visual feedback. Videos are available at https://taochenshh.github.io/projects/tactofind.