# Deep Visuo-Tactile Learning: Estimation of Tactile Properties from Images

arXiv: https://arxiv.org/abs/1803.03435 · HTML: https://mlsanity.com/paper/1803.03435/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Mar 2018.

## Built on by (most cited first, 9 of 9)

- [Visual-Tactile Cross-Modal Data Generation using Residue-Fusion GAN with Feature-Matching and Perceptual Losses](https://mlsanity.com/paper/2107.05468/) (2021)
- [Telextiles: End-to-end Remote Transmission of Fabric Tactile Sensation](https://mlsanity.com/paper/2405.03363/) (2024)
- [Multimodal perception for dexterous manipulation](https://mlsanity.com/paper/2112.14298/) (2021)
- [Metarobotics for Industry and Society: Vision, Technologies, and Opportunities](https://mlsanity.com/paper/2404.00797/) (2024)
- [Visuo-Tactile Zero-Shot Object Recognition with Vision-Language Model](https://mlsanity.com/paper/2409.09276/) (2024)
- [Image-to-Force Estimation for Soft Tissue Interaction in Robotic-Assisted Surgery Using Structured Light](https://mlsanity.com/paper/2501.08593/) (2025)
- [Material Classification Using Active Temperature Controllable Robotic Gripper](https://mlsanity.com/paper/2111.15344/) (2021)
- [Physical Context and Timing Aware Sequence Generating GANs](https://mlsanity.com/paper/2110.04077/) (2021)
- [FusionSense: Tri-Stage Near-Sensor Learning for Runtime-Adaptive Multimodal Edge Intelligence](https://mlsanity.com/paper/2605.22868/) (2026)

## Abstract (arXiv)

Estimation of tactile properties from vision, such as slipperiness or roughness, is important to effectively interact with the environment. These tactile properties help us decide which actions we should choose and how to perform them. E.g., we can drive slower if we see that we have bad traction or grasp tighter if an item looks slippery. We believe that this ability also helps robots to enhance their understanding of the environment, and thus enables them to tailor their actions to the situation at hand. We therefore propose a model to estimate the degree of tactile properties from visual perception alone (e.g., the level of slipperiness or roughness). Our method extends a encoder-decoder network, in which the latent variables are visual and tactile features. In contrast to previous works, our method does not require manual labeling, but only RGB images and the corresponding tactile sensor data. All our data is collected with a webcam and uSkin tactile sensor mounted on the end-effector of a Sawyer robot, which strokes the surfaces of 25 different materials. We show that our model generalizes to materials not included in the training data by evaluating the feature space, indicating that it has learned to associate important tactile properties with images.