# Bayesian and Neural Inference on LSTM-based Object Recognition from Tactile and Kinesthetic Information

arXiv: https://arxiv.org/abs/2306.06423 · HTML: https://mlsanity.com/paper/2306.06423/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 53 references that are in the map.
- First posted on arXiv 10 Jun 2023.

## Builds on (most cited first, 8 of 8)

- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder](https://mlsanity.com/paper/1711.00614/) (2017)
- [Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks](https://mlsanity.com/paper/1907.13098/) (2019)
- [Robotic Tactile Perception of Object Properties: A Review](https://mlsanity.com/paper/1711.03810/) (2017)
- [Deep Learning for Tactile Understanding From Visual and Haptic Data](https://mlsanity.com/paper/1511.06065/) (2015)
- [iCLAP: Shape Recognition by Combining Proprioception and Touch Sensing](https://mlsanity.com/paper/1806.07507/) (2018)
- [HANDS: A Multimodal Dataset for Modeling Towards Human Grasp Intent Inference in Prosthetic Hands](https://mlsanity.com/paper/2103.04845/) (2021)
- [CNN-based Methods for Object Recognition with High-Resolution Tactile Sensors](https://mlsanity.com/paper/2305.12417/) (2023)

## Built on by (most cited first, 3 of 3)

- [Generalize by Touching: Tactile Ensemble Skill Transfer for Robotic Furniture Assembly](https://mlsanity.com/paper/2404.17684/) (2024)
- [Fine-Tuning Generative Models as an Inference Method for Robotic Tasks](https://mlsanity.com/paper/2310.12862/) (2023)
- [Multistatic-Radar RCS-Signature Recognition of Aerial Vehicles: A Bayesian Fusion Approach](https://mlsanity.com/paper/2402.17987/) (2024)

## Abstract (arXiv)

Recent advances in the field of intelligent robotic manipulation pursue providing robotic hands with touch sensitivity. Haptic perception encompasses the sensing modalities encountered in the sense of touch (e.g., tactile and kinesthetic sensations). This letter focuses on multimodal object recognition and proposes analytical and data-driven methodologies to fuse tactile- and kinesthetic-based classification results. The procedure is as follows: a three-finger actuated gripper with an integrated high-resolution tactile sensor performs squeeze-and-release Exploratory Procedures (EPs). The tactile images and kinesthetic information acquired using angular sensors on the finger joints constitute the time-series datasets of interest. Each temporal dataset is fed to a Long Short-term Memory (LSTM) Neural Network, which is trained to classify in-hand objects. The LSTMs provide an estimation of the posterior probability of each object given the corresponding measurements, which after fusion allows to estimate the object through Bayesian and Neural inference approaches. An experiment with 36-classes is carried out to evaluate and compare the performance of the fused, tactile, and kinesthetic perception systems.The results show that the Bayesian-based classifiers improves capabilities for object recognition and outperforms the Neural-based approach.