mlsanityData to 14 Oct 2026

Sparsh: Self-supervised touch representations for vision-based tactile sensing

arXiv 2410.24090 ↗First posted 31 Oct 2024cs.ROcited by 64

Not on the map. Why?

Abstract (arXiv)

In this work, we introduce general purpose touch representations for the increasingly accessible class of vision-based tactile sensors. Such sensors have led to many recent advances in robot manipulation as they markedly complement vision, yet solutions today often rely on task and sensor specific handcrafted perception models. Collecting real data at scale with task centric ground truth labels, like contact forces and slip, is a challenge further compounded by sensors of various form factor differing in aspects like lighting and gel markings. To tackle this we turn to self-supervised learning (SSL) that has demonstrated remarkable performance in computer vision. We present Sparsh, a family of SSL models that can support various vision-based tactile sensors, alleviating the need for custom labels through pre-training on 460k+ tactile images with masking and self-distillation in pixel and latent spaces. We also build TacBench, to facilitate standardized benchmarking across sensors and models, comprising of six tasks ranging from comprehending tactile properties to enabling physical perception and manipulation planning. In evaluations, we find that SSL pre-training for touch representation outperforms task and sensor-specific end-to-end training by 95.1% on average over TacBench, and Sparsh (DINO) and Sparsh (IJEPA) are the most competitive, indicating the merits of learning in latent space for tactile images. Project page: https://sparsh-ssl.github.io/

Key facts. Cited by 64 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 Oct 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 20; 2026: 44.

64 papers.

Sort:

By area:

  1. 2025Embodied AI Agents: Modeling the Worldcited by 61
  2. 2025AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensorscited by 48
  3. 2025Failure Prediction at Runtime for Generative Robot Policiescited by 42
  4. 2025Multimodal Fusion and Vision-Language Models: A Survey for Robot Visioncited by 40
  5. 2025VLA-Touch: Enhancing Vision-Language-Action Models with Dual-Level Tactile Feedbackcited by 38
  6. 2025Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Grippercited by 29
  7. 2025Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulationcited by 18
  8. 2026VTAM: Video-Tactile-Action Models for Complex Physical Interaction Beyond VLAscited by 17
  9. 2025A Survey on Imitation Learning for Contact-Rich Tasks in Roboticscited by 14
  10. 2026AnyTouch 2: General Optical Tactile Representation Learning For Dynamic Tactile Perceptioncited by 12
  11. 2026Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attentioncited by 10
  12. 2026AT-VLA: Adaptive Tactile Injection for Enhanced Feedback Reaction in Vision-Language-Action Modelscited by 9
38 more
  1. 2025ACT-JEPA: Novel Joint-Embedding Predictive Architecture for Efficient Policy Representation Learningcited by 8
  2. 2025Sensor-Invariant Tactile Representationcited by 8
  3. 2025Universal Visuo-Tactile Video Understanding for Embodied Interactioncited by 8
  4. 2026Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLAcited by 5
  5. 2026Tactile Modality Fusion for Vision-Language-Action Modelscited by 4
  6. 2026ContactWorld: What Representations Matter for Vision-Tactile Latent World Models in Contact-Rich Manipulationcited by 4
  7. 2025SToLa: Self-Adaptive Touch-Language Framework with Tactile Commonsense Reasoning in Open-Ended Scenarioscited by 3
  8. 2025Tactile MNIST: Benchmarking Active Tactile Perceptioncited by 3
  9. 2025RoboManipBaselines: A Unified Framework for Imitation Learning in Robotic Manipulation across Real and Simulation Environmentscited by 3
  10. 2026TacCoRL: Integrating Tactile Feedback into VLA via Simulationcited by 3
  11. 2026What Can Latent World Models Know? Physical Parameter Identifiability in Multimodal Predictive Representationscited by 3
  12. 2025Cross-Sensor Touch Generationcited by 2
  13. 2025Collaborative Representation Learning for Alignment of Tactile, Language, and Vision Modalitiescited by 2
  14. 2026TaCo: A Benchmark for Lossless and Lossy Codecs of Heterogeneous Tactile Datacited by 2
  15. 2026TouchThinker: Scaling Tactile Commonsense Reasoning to the Open World with Large-scale Data and Action-aware Representationcited by 2
  16. 2026Imitation from Heterogeneous Demonstrations using Grounded Latent-Action World Modelscited by 2
  17. 2026Seeing Touch from Motion: A Unified Modality-Aware Visuo-Tactile Policy with Tactile Motion Correlationcited by 2
  18. 2026TacForcing: Streaming Action Generation with Execution-Time Tactile Feedbackcited by 2
  19. 2026TacPAC: Tactile Prediction and Real-Time Action Correction in World-Action Models for Contact-Rich Manipulationcited by 2
  20. 2026Bench2Dex: Benchmarking Visuo-Tactile Bimanual Dexterous Manipulation Across Dexterous Handscited by 2
  21. 2025Toward Artificial Palpation: Representation Learning of Touch on Soft Bodiescited by 1
  22. 2026FlowTouch: View-Invariant Visuo-Tactile Predictioncited by 1
  23. 2026Touch-R1: Reinforcing Touch Reasoning in MLLMscited by 1
  24. 2026OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policiescited by 1
  25. 2026Kepler-Encoder-v0.1: Towards a Multimodal Embedding Model for Robotscited by 1
  26. 2026TouchSight: Bare-Handed Tactile Prediction from Egocentric Video via Generative Visual Augmentationcited by 1
  27. 2025ControlTac: Scaling Tactile Data with Physically Controlled Tactile Image Generationcited by 0
  28. 2025D3Grasp: Diverse and Deformable Dexterous Grasping for General Objectscited by 0
  29. 2025SARL: Spatially-Aware Self-Supervised Representation Learning for Visuo-Tactile Perceptioncited by 0
  30. 2026Tactile Memory with Soft Robot: Robust Object Insertion via Masked Encoding and Soft Wristcited by 0
  31. 2026Let Robots Feel Your Touch: Visuo-Tactile Cortical Alignment for Embodied Mirror Resonancecited by 0
  32. 2026More with LESS -- Local Scene Representations for Tactile Imagingcited by 0
  33. 2026GeoTLM: Geometry-aware Tactile-Language Models for Contact Motion Orientation Reasoning of Dynamic Objectscited by 0
  34. 2026RCT: A Robot-Collected Touch-Vision-Language Dataset for Tactile Generalizationcited by 0
  35. 2026Tactus: Open-Vocabulary Object Recognition from Low-Cost Pressure Arrayscited by 0
  36. 2026ViTaR: Visuo-Tactile Residual Adaptation for Foundation VLA Manipulationcited by 0
  37. 2026Zero-Shot Transfer of Force Map Estimation Across GelSight Mini Sensorscited by 0
  38. 2026VT-MUSE: Multimodal Unified Sequential Visuotactile Representation Learning for Manipulationcited by 0

50 most cited of 64 All 64 (JSON)