# Action Conditioned Tactile Prediction: case study on slip prediction

arXiv: https://arxiv.org/abs/2205.09430 · HTML: https://mlsanity.com/paper/2205.09430/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 39 references that are in the map.
- First posted on arXiv 19 May 2022.

## Builds on (most cited first, 16 of 16)

- [Unsupervised Learning for Physical Interaction through Video Prediction](https://mlsanity.com/paper/1605.07157/) (2016)
- [Stochastic Video Generation with a Learned Prior](https://mlsanity.com/paper/1802.07687/) (2018)
- [Stochastic Variational Video Prediction](https://mlsanity.com/paper/1710.11252/) (2017)
- [Stochastic Adversarial Video Prediction](https://mlsanity.com/paper/1804.01523/) (2018)
- [A Random Forest Guided Tour](https://mlsanity.com/paper/1511.05741/) (2015)
- [Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks](https://mlsanity.com/paper/1810.10191/) (2018)
- [More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch](https://mlsanity.com/paper/1805.11085/) (2018)
- [High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks](https://mlsanity.com/paper/1911.01655/) (2019)
- [Manipulation by Feel: Touch-Based Control with Deep Predictive Models](https://mlsanity.com/paper/1903.04128/) (2019)
- [Tactile Dexterity: Manipulation Primitives with Tactile Feedback](https://mlsanity.com/paper/2002.03236/) (2020)
- [MAT: Multi-Fingered Adaptive Tactile Grasping via Deep Reinforcement Learning](https://mlsanity.com/paper/1909.04787/) (2019)
- [Action-conditioned Benchmarking of Robotic Video Prediction Models: a Comparative Study](https://mlsanity.com/paper/1910.02564/) (2019)
- [Towards Learning to Detect and Predict Contact Events on Vision-based Tactile Sensors](https://mlsanity.com/paper/1910.03973/) (2019)
- [Robust Learning of Tactile Force Estimation through Robot Interaction](https://mlsanity.com/paper/1810.06187/) (2018)
- [FingerVision Tactile Sensor Design and Slip Detection Using Convolutional LSTM Network](https://mlsanity.com/paper/1810.02653/) (2018)
- [Non-Matrix Tactile Sensors: How Can Be Exploited Their Local Connectivity For Predicting Grasp Stability?](https://mlsanity.com/paper/1809.05551/) (2018)

## Built on by (most cited first, 8 of 8)

- [TaF-VLA: Tactile-Force Alignment in Vision-Language-Action Models for Force-aware Manipulation](https://mlsanity.com/paper/2601.20321/) (2026)
- [UniTacVLA: Unified Tactile Understanding and Prediction in Vision Language Action Models](https://mlsanity.com/paper/2606.31723/) (2026)
- [Autonomous Strawberry Picking Robotic System (Robofruit)](https://mlsanity.com/paper/2301.03947/) (2023)
- [Investigating Sensors and Methods in Grasp State Classification in Agricultural Manipulation](https://mlsanity.com/paper/2508.11588/) (2025)
- [Deep Functional Predictive Control for Strawberry Cluster Manipulation using Tactile Prediction](https://mlsanity.com/paper/2303.05393/) (2023)
- [Multi-Modal World Model for Physical Robot Interactions: Simultaneous Visual and Tactile Predictions for Enhanced Accuracy](https://mlsanity.com/paper/2304.11193/) (2023)
- [Single and bi-layered 2-D acoustic soft tactile skin (AST2)](https://mlsanity.com/paper/2401.14292/) (2024)
- [TacSushi: Tactile-Grounded World-Action Modeling for Dexterous Sushi Manipulation](https://mlsanity.com/paper/2609.19613/) (2026)

## Abstract (arXiv)

Tactile predictive models can be useful across several robotic manipulation tasks, e.g. robotic pushing, robotic grasping, slip avoidance, and in-hand manipulation. However, available tactile prediction models are mostly studied for image-based tactile sensors and there is no comparison study indicating the best performing models. In this paper, we presented two novel data-driven action-conditioned models for predicting tactile signals during real-world physical robot interaction tasks (1) action condition tactile prediction and (2) action conditioned tactile-video prediction models. We use a magnetic-based tactile sensor that is challenging to analyse and test state-of-the-art predictive models and the only existing bespoke tactile prediction model. We compare the performance of these models with those of our proposed models. We perform the comparison study using our novel tactile-enabled dataset containing 51,000 tactile frames of a real-world robotic manipulation task with 11 flat-surfaced household objects. Our experimental results demonstrate the superiority of our proposed tactile prediction models in terms of qualitative, quantitative and slip prediction scores.