# SARL: Spatially-Aware Self-Supervised Representation Learning for Visuo-Tactile Perception

arXiv: https://arxiv.org/abs/2512.01908 · HTML: https://mlsanity.com/paper/2512.01908/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 1 Dec 2025.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [Self-Supervised Representation Learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2110.09327/) (2021)
- [Touch and Go: Learning from Human-Collected Vision and Touch](https://mlsanity.com/paper/2211.12498/) (2022)
- [Sparsh: Self-supervised touch representations for vision-based tactile sensing](https://mlsanity.com/paper/2410.24090/) (2024)
- [AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors](https://mlsanity.com/paper/2502.12191/) (2025)
- [Transferable Tactile Transformers for Representation Learning Across Diverse Sensors and Tasks](https://mlsanity.com/paper/2406.13640/) (2024)
- [Multimodal Visual-Tactile Representation Learning through Self-Supervised Contrastive Pre-Training](https://mlsanity.com/paper/2401.12024/) (2024)
- [Deep Learning for Tactile Understanding From Visual and Haptic Data](https://mlsanity.com/paper/1511.06065/) (2015)
- [UniT: Data Efficient Tactile Representation with Generalization to Unseen Objects](https://mlsanity.com/paper/2408.06481/) (2024)
- [A Survey on Self-Supervised Representation Learning](https://mlsanity.com/paper/2308.11455/) (2023)
- [Collapse-Proof Non-Contrastive Self-Supervised Learning](https://mlsanity.com/paper/2410.04959/) (2024)
- [ViTacTip: Design and Verification of a Novel Biomimetic Physical Vision-Tactile Fusion Sensor](https://mlsanity.com/paper/2402.00199/) (2024)
- [Representation Learning via Non-Contrastive Mutual Information](https://mlsanity.com/paper/2504.16667/) (2025)
- [Tac-VGNN: A Voronoi Graph Neural Network for Pose-Based Tactile Servoing](https://mlsanity.com/paper/2303.02708/) (2023)
- [Attention for Robot Touch: Tactile Saliency Prediction for Robust Sim-to-Real Tactile Control](https://mlsanity.com/paper/2307.14510/) (2023)
- [MagicTac: A Novel High-Resolution 3D Multi-layer Grid-Based Tactile Sensor](https://mlsanity.com/paper/2402.01366/) (2024)
- [TacPalm: A Soft Gripper with a Biomimetic Optical Tactile Palm for Stable Precise Grasping](https://mlsanity.com/paper/2409.15239/) (2024)
- [Design and Benchmarking of A Multi-Modality Sensor for Robotic Manipulation with GAN-Based Cross-Modality Interpretation](https://mlsanity.com/paper/2501.02303/) (2025)

## Abstract (arXiv)

Contact-rich robotic manipulation requires representations that encode local geometry. Vision provides global context but lacks direct measurements of properties such as texture and hardness, whereas touch supplies these cues. Modern visuo-tactile sensors capture both modalities in a single fused image, yielding intrinsically aligned inputs that are well suited to manipulation tasks requiring visual and tactile information. Most self-supervised learning (SSL) frameworks, however, compress feature maps into a global vector, discarding spatial structure and misaligning with the needs of manipulation. To address this, we propose SARL, a spatially-aware SSL framework that augments the Bootstrap Your Own Latent (BYOL) architecture with three map-level objectives, including Saliency Alignment (SAL), Patch-Prototype Distribution Alignment (PPDA), and Region Affinity Matching (RAM), to keep attentional focus, part composition, and geometric relations consistent across views. These losses act on intermediate feature maps, complementing the global objective. SARL consistently outperforms nine SSL baselines across six downstream tasks with fused visual-tactile data. On the geometry-sensitive edge-pose regression task, SARL achieves a Mean Absolute Error (MAE) of 0.3955, a 30% relative improvement over the next-best SSL method (0.5682 MAE) and approaching the supervised upper bound. These findings indicate that, for fused visual-tactile data, the most effective signal is structured spatial equivariance, in which features vary predictably with object geometry, which enables more capable robotic perception.