# Seeing Touch from Motion: A Unified Modality-Aware Visuo-Tactile Policy with Tactile Motion Correlation

arXiv: https://arxiv.org/abs/2606.29941 · HTML: https://mlsanity.com/paper/2606.29941/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 72 references that are in the map.
- First posted on arXiv 29 Jun 2026.

## Builds on (most cited first, 30 of 58)

- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [Emerging Properties in Unified Multimodal Pretraining](https://mlsanity.com/paper/2505.14683/) (2025)
- [Causal World Modeling for Robot Control](https://mlsanity.com/paper/2601.21998/) (2026)
- [Motus: A Unified Latent Action World Model](https://mlsanity.com/paper/2512.13030/) (2025)
- [SAM 3D: 3Dfy Anything in Images](https://mlsanity.com/paper/2511.16624/) (2025)
- [Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models](https://mlsanity.com/paper/2411.04996/) (2024)
- [ALOHA Unleashed: A Simple Recipe for Robot Dexterity](https://mlsanity.com/paper/2410.13126/) (2024)
- [Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation](https://mlsanity.com/paper/2503.02881/) (2025)
- [A Pragmatic VLA Foundation Model](https://mlsanity.com/paper/2601.18692/) (2026)
- [ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation](https://mlsanity.com/paper/2505.22159/) (2025)
- [3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing](https://mlsanity.com/paper/2410.24091/) (2024)
- [Fast Optical Flow using Dense Inverse Search](https://mlsanity.com/paper/1603.03590/) (2016)
- [See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation](https://mlsanity.com/paper/2212.03858/) (2022)
- [Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalization](https://mlsanity.com/paper/2601.12993/) (2026)
- [Sparsh: Self-supervised touch representations for vision-based tactile sensing](https://mlsanity.com/paper/2410.24090/) (2024)
- [VTLA: Vision-Tactile-Language-Action Model with Preference Learning for Insertion Manipulation](https://mlsanity.com/paper/2505.09577/) (2025)
- [InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulation](https://mlsanity.com/paper/2601.02456/) (2026)
- [AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors](https://mlsanity.com/paper/2502.12191/) (2025)
- [Visuo-Tactile Transformers for Manipulation](https://mlsanity.com/paper/2210.00121/) (2022)
- [GelSlim3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Finger](https://mlsanity.com/paper/2103.12269/) (2021)
- [VLA-Touch: Enhancing Vision-Language-Action Models with Dual-Level Tactile Feedback](https://mlsanity.com/paper/2507.17294/) (2025)
- [OmniVTLA: Vision-Tactile-Language-Action Models with Semantic-Aligned Tactile Sensing](https://mlsanity.com/paper/2508.08706/) (2025)
- [ReSkin: versatile, replaceable, lasting tactile skins](https://mlsanity.com/paper/2111.00071/) (2021)
- [MimicTouch: Leveraging Multi-modal Human Tactile Demonstrations for Contact-rich Manipulation](https://mlsanity.com/paper/2310.16917/) (2023)
- [FACTR: Force-Attending Curriculum Training for Contact-Rich Policy Learning](https://mlsanity.com/paper/2502.17432/) (2025)
- [FoAR: Force-Aware Reactive Policy for Contact-Rich Robotic Manipulation](https://mlsanity.com/paper/2411.15753/) (2024)
- [Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper](https://mlsanity.com/paper/2507.15062/) (2025)
- [ViTaMIn: Learning Contact-Rich Tasks Through Robot-Free Visuo-Tactile Manipulation Interface](https://mlsanity.com/paper/2504.06156/) (2025)
- [MLA: A Multisensory Language-Action Model for Multimodal Understanding and Forecasting in Robotic Manipulation](https://mlsanity.com/paper/2509.26642/) (2025)

## Built on by (most cited first, 2 of 2)

- [τ: Learning Touch-Augmented Vision-Language-Action Models from Future Visual Supervision](https://mlsanity.com/paper/2607.24485/) (2026)
- [N0-Foundation: Towards the Age of Tactile Intelligence](https://mlsanity.com/paper/2608.29601/) (2026)

## Abstract (arXiv)

Visuo-Tactile policies leveraging optical tactile sensors have shown great promise in contact-rich manipulation. These sensors achieve high spatial resolution and multi-dimensional force sensing by utilizing an internal camera to monitor the deformation of their elastic gel surface, thereby indirectly inferring tactile cues. Despite their advantages, extracting fine-grained contact states necessary for contact-rich manipulation remains an open challenge. Existing methods typically use either raw images or cumulative motion fields to represent tactile cues. However, both are prone to perception ambiguity. Raw tactile images mainly capture appearance changes, while cumulative motion fields only reflect the aggregate gel deformation. Consequently, distinct fine-grained contact states can exhibit highly similar patterns, making it difficult to explicitly distinguish subtle contact variations. To address this issue, we explore the dynamic priors of tactile motion and discover that the correlation between transient and cumulative motion can explicitly distinguish fine-grained contact states. Based on this insight, we propose a motion-aware tactile representation to facilitate contact-rich manipulation. Beyond tactile representation, effective fusion of tactile and visual modalities is also critical. Most existing fusion methods either directly concatenate features from each modality or train modality-specific networks separately and fuse their outputs. However, these strategies struggle to simultaneously model cross-modal interactions and preserve modality-specific characteristics. In this work, we take advantage of the Mixture-of-Transformers architecture and propose a unified modality-aware visuo-tactile policy that captures cross-modal complementarity while maintaining modality-specific properties.