# Extended Tactile Perception: Vibration Sensing through Tools and Grasped Objects

arXiv: https://arxiv.org/abs/2106.00489 · HTML: https://mlsanity.com/paper/2106.00489/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 51 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 6 of 6)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [Event-Driven Visual-Tactile Sensing and Learning for Robots](https://mlsanity.com/paper/2009.07083/) (2020)
- [Robotic Tactile Perception of Object Properties: A Review](https://mlsanity.com/paper/1711.03810/) (2017)
- [Towards Robotic Feeding: Role of Haptics in Fork-based Food Manipulation](https://mlsanity.com/paper/1804.08768/) (2018)
- [Factorized Inference in Deep Markov Models for Incomplete Multimodal Time Series](https://mlsanity.com/paper/1905.13570/) (2019)

## Built on by (most cited first, 4 of 4)

- [Evetac: An Event-based Optical Tactile Sensor for Robotic Manipulation](https://mlsanity.com/paper/2312.01236/) (2023)
- [Optimizing BioTac Simulation for Realistic Tactile Perception](https://mlsanity.com/paper/2404.10425/) (2024)
- [Event-Driven Tactile Learning with Location Spiking Neurons](https://mlsanity.com/paper/2209.01080/) (2022)
- [Boost Event-Driven Tactile Learning with Location Spiking Neurons](https://mlsanity.com/paper/2210.04277/) (2022)

## Abstract (arXiv)

Humans display the remarkable ability to sense the world through tools and other held objects. For example, we are able to pinpoint impact locations on a held rod and tell apart different textures using a rigid probe. In this work, we consider how we can enable robots to have a similar capacity, i.e., to embody tools and extend perception using standard grasped objects. We propose that vibro-tactile sensing using dynamic tactile sensors on the robot fingers, along with machine learning models, enables robots to decipher contact information that is transmitted as vibrations along rigid objects. This paper reports on extensive experiments using the BioTac micro-vibration sensor and a new event dynamic sensor, the NUSkin, capable of multi-taxel sensing at 4~kHz. We demonstrate that fine localization on a held rod is possible using our approach (with errors less than 1 cm on a 20 cm rod). Next, we show that vibro-tactile perception can lead to reasonable grasp stability prediction during object handover, and accurate food identification using a standard fork. We find that multi-taxel vibro-tactile sensing at sufficiently high sampling rate led to the best performance across the various tasks and objects. Taken together, our results provides both evidence and guidelines for using vibro-tactile perception to extend tactile perception, which we believe will lead to enhanced competency with tools and better physical human-robot-interaction.