# Seeing Through your Skin: Recognizing Objects with a Novel Visuotactile Sensor

arXiv: https://arxiv.org/abs/2011.09552 · HTML: https://mlsanity.com/paper/2011.09552/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2020.

## Built on by (most cited first, 8 of 8)

- [DenseTact: Optical Tactile Sensor for Dense Shape Reconstruction](https://mlsanity.com/paper/2201.01367/) (2022)
- [AcTExplore: Active Tactile Exploration of Unknown Objects](https://mlsanity.com/paper/2310.08745/) (2023)
- [Multimodal and Force-Matched Imitation Learning with a See-Through Visuotactile Sensor](https://mlsanity.com/paper/2311.01248/) (2023)
- [Learning Intuitive Physics with Multimodal Generative Models](https://mlsanity.com/paper/2101.04454/) (2021)
- [Sim2Real Bilevel Adaptation for Object Surface Classification using Vision-Based Tactile Sensors](https://mlsanity.com/paper/2311.01380/) (2023)
- [Tactile Modality Fusion for Vision-Language-Action Models](https://mlsanity.com/paper/2603.14604/) (2026)
- [Tactile-based Multimodal Fusion in Embodied Intelligence: A Survey of Vision, Language, and Contact-Driven Paradigms](https://mlsanity.com/paper/2605.17336/) (2026)
- [Seeing Touch from Motion: A Unified Modality-Aware Visuo-Tactile Policy with Tactile Motion Correlation](https://mlsanity.com/paper/2606.29941/) (2026)

## Abstract (arXiv)

We introduce a new class of vision-based sensor and associated algorithmic processes that combine visual imaging with high-resolution tactile sending, all in a uniform hardware and computational architecture. We demonstrate the sensor's efficacy for both multi-modal object recognition and metrology. Object recognition is typically formulated as an unimodal task, but by combining two sensor modalities we show that we can achieve several significant performance improvements. This sensor, named the See-Through-your-Skin sensor (STS), is designed to provide rich multi-modal sensing of contact surfaces. Inspired by recent developments in optical tactile sensing technology, we address a key missing feature of these sensors: the ability to capture a visual perspective of the region beyond the contact surface. Whereas optical tactile sensors are typically opaque, we present a sensor with a semitransparent skin that has the dual capabilities of acting as a tactile sensor and/or as a visual camera depending on its internal lighting conditions. This paper details the design of the sensor, showcases its dual sensing capabilities, and presents a deep learning architecture that fuses vision and touch. We validate the ability of the sensor to classify household objects, recognize fine textures, and infer their physical properties both through numerical simulations and experiments with a smart countertop prototype.