# Sparsh: Self-supervised touch representations for vision-based tactile sensing

arXiv: https://arxiv.org/abs/2410.24090 · HTML: https://mlsanity.com/paper/2410.24090/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2024.

## Built on by (most cited first, 30 of 64)

- [Embodied AI Agents: Modeling the World](https://mlsanity.com/paper/2506.22355/) (2025)
- [AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors](https://mlsanity.com/paper/2502.12191/) (2025)
- [Failure Prediction at Runtime for Generative Robot Policies](https://mlsanity.com/paper/2510.09459/) (2025)
- [Multimodal Fusion and Vision-Language Models: A Survey for Robot Vision](https://mlsanity.com/paper/2504.02477/) (2025)
- [VLA-Touch: Enhancing Vision-Language-Action Models with Dual-Level Tactile Feedback](https://mlsanity.com/paper/2507.17294/) (2025)
- [Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper](https://mlsanity.com/paper/2507.15062/) (2025)
- [Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulation](https://mlsanity.com/paper/2512.23864/) (2025)
- [VTAM: Video-Tactile-Action Models for Complex Physical Interaction Beyond VLAs](https://mlsanity.com/paper/2603.23481/) (2026)
- [A Survey on Imitation Learning for Contact-Rich Tasks in Robotics](https://mlsanity.com/paper/2506.13498/) (2025)
- [AnyTouch 2: General Optical Tactile Representation Learning For Dynamic Tactile Perception](https://mlsanity.com/paper/2602.09617/) (2026)
- [Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attention](https://mlsanity.com/paper/2606.26663/) (2026)
- [AT-VLA: Adaptive Tactile Injection for Enhanced Feedback Reaction in Vision-Language-Action Models](https://mlsanity.com/paper/2605.07308/) (2026)
- [ACT-JEPA: Novel Joint-Embedding Predictive Architecture for Efficient Policy Representation Learning](https://mlsanity.com/paper/2501.14622/) (2025)
- [Sensor-Invariant Tactile Representation](https://mlsanity.com/paper/2502.19638/) (2025)
- [Universal Visuo-Tactile Video Understanding for Embodied Interaction](https://mlsanity.com/paper/2505.22566/) (2025)
- [Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLA](https://mlsanity.com/paper/2603.08122/) (2026)
- [Tactile Modality Fusion for Vision-Language-Action Models](https://mlsanity.com/paper/2603.14604/) (2026)
- [ContactWorld: What Representations Matter for Vision-Tactile Latent World Models in Contact-Rich Manipulation](https://mlsanity.com/paper/2606.13877/) (2026)
- [SToLa: Self-Adaptive Touch-Language Framework with Tactile Commonsense Reasoning in Open-Ended Scenarios](https://mlsanity.com/paper/2505.04201/) (2025)
- [Tactile MNIST: Benchmarking Active Tactile Perception](https://mlsanity.com/paper/2506.06361/) (2025)
- [RoboManipBaselines: A Unified Framework for Imitation Learning in Robotic Manipulation across Real and Simulation Environments](https://mlsanity.com/paper/2509.17057/) (2025)
- [TacCoRL: Integrating Tactile Feedback into VLA via Simulation](https://mlsanity.com/paper/2606.11743/) (2026)
- [What Can Latent World Models Know? Physical Parameter Identifiability in Multimodal Predictive Representations](https://mlsanity.com/paper/2607.27017/) (2026)
- [Cross-Sensor Touch Generation](https://mlsanity.com/paper/2510.09817/) (2025)
- [Collaborative Representation Learning for Alignment of Tactile, Language, and Vision Modalities](https://mlsanity.com/paper/2511.11512/) (2025)
- [TaCo: A Benchmark for Lossless and Lossy Codecs of Heterogeneous Tactile Data](https://mlsanity.com/paper/2602.09893/) (2026)
- [TouchThinker: Scaling Tactile Commonsense Reasoning to the Open World with Large-scale Data and Action-aware Representation](https://mlsanity.com/paper/2606.11637/) (2026)
- [Imitation from Heterogeneous Demonstrations using Grounded Latent-Action World Models](https://mlsanity.com/paper/2606.21672/) (2026)
- [Seeing Touch from Motion: A Unified Modality-Aware Visuo-Tactile Policy with Tactile Motion Correlation](https://mlsanity.com/paper/2606.29941/) (2026)
- [TacForcing: Streaming Action Generation with Execution-Time Tactile Feedback](https://mlsanity.com/paper/2608.25798/) (2026)

## Abstract (arXiv)

In this work, we introduce general purpose touch representations for the increasingly accessible class of vision-based tactile sensors. Such sensors have led to many recent advances in robot manipulation as they markedly complement vision, yet solutions today often rely on task and sensor specific handcrafted perception models. Collecting real data at scale with task centric ground truth labels, like contact forces and slip, is a challenge further compounded by sensors of various form factor differing in aspects like lighting and gel markings. To tackle this we turn to self-supervised learning (SSL) that has demonstrated remarkable performance in computer vision. We present Sparsh, a family of SSL models that can support various vision-based tactile sensors, alleviating the need for custom labels through pre-training on 460k+ tactile images with masking and self-distillation in pixel and latent spaces. We also build TacBench, to facilitate standardized benchmarking across sensors and models, comprising of six tasks ranging from comprehending tactile properties to enabling physical perception and manipulation planning. In evaluations, we find that SSL pre-training for touch representation outperforms task and sensor-specific end-to-end training by 95.1% on average over TacBench, and Sparsh (DINO) and Sparsh (IJEPA) are the most competitive, indicating the merits of learning in latent space for tactile images. Project page: https://sparsh-ssl.github.io/