# Deep Learning for Surface Material Classification Using Haptic And Visual Information

arXiv: https://arxiv.org/abs/1512.06658 · HTML: https://mlsanity.com/paper/1512.06658/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2015.

## Built on by (most cited first, 6 of 6)

- [DPF: Learning Dense Prediction Fields with Weak Supervision](https://mlsanity.com/paper/2303.16890/) (2023)
- [CNN-based Methods for Object Recognition with High-Resolution Tactile Sensors](https://mlsanity.com/paper/2305.12417/) (2023)
- [A Deep Learning Approach for Pixel-level Material Classification via Hyperspectral Imaging](https://mlsanity.com/paper/2409.13498/) (2024)
- [Incorporating simulated spatial context information improves the effectiveness of contrastive learning models](https://mlsanity.com/paper/2401.15120/) (2024)
- [Multimodal Multi-User Surface Recognition with the Kernel Two-Sample Test](https://mlsanity.com/paper/2303.04930/) (2023)
- [Living in a Material World: Learning Material Properties from Full-Waveform Flash Lidar Data for Semantic Segmentation](https://mlsanity.com/paper/2305.04334/) (2023)

## Abstract (arXiv)

When a user scratches a hand-held rigid tool across an object surface, an acceleration signal can be captured, which carries relevant information about the surface. More importantly, such a haptic signal is complementary to the visual appearance of the surface, which suggests the combination of both modalities for the recognition of the surface material. In this paper, we present a novel deep learning method dealing with the surface material classification problem based on a Fully Convolutional Network (FCN), which takes as input the aforementioned acceleration signal and a corresponding image of the surface texture. Compared to previous surface material classification solutions, which rely on a careful design of hand-crafted domain-specific features, our method automatically extracts discriminative features utilizing the advanced deep learning methodologies. Experiments performed on the TUM surface material database demonstrate that our method achieves state-of-the-art classification accuracy robustly and efficiently.