ViTaS: Visual Tactile Soft Fusion Contrastive Learning for Visuomotor Learning
Not on the map. Why?
Abstract (arXiv)
Tactile information plays a crucial role in human manipulation tasks and has recently garnered increasing attention in robotic manipulation. However, existing approaches mostly focus on the alignment of visual and tactile features and the integration mechanism tends to be direct concatenation. Consequently, they struggle to effectively cope with occluded scenarios due to neglecting the inherent complementary nature of both modalities and the alignment may not be exploited enough, limiting the potential of their real-world deployment. In this paper, we present ViTaS, a simple yet effective framework that incorporates both visual and tactile information to guide the behavior of an agent. We introduce Soft Fusion Contrastive Learning, an advanced version of conventional contrastive learning method and a CVAE module to utilize the alignment and complementarity within visuo-tactile representations. We demonstrate the effectiveness of our method in 12 simulated and 3 real-world environments, and our experiments show that ViTaS significantly outperforms existing baselines. Project page: https://skyrainwind.github.io/ViTaS/index.html.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 32 of its 39 references that are in the map. First posted on arXiv 12 Feb 2026.
Builds on
32 of 39 references.
References · 32
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
- 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
- 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
- 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
- 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
- 2018Visual Reinforcement Learning with Imagined Goalscited by 248
- 2020Self-supervised Co-training for Video Representation Learningcited by 232
- 2018Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Taskscited by 150
- 2021Multimodal Contrastive Training for Visual Representation Learningcited by 122
- 2022In-Hand Object Rotation via Rapid Motor Adaptationcited by 102
20 more
- 2024Binding Touch to Everything: Learning Unified Multimodal Tactile Representationscited by 94
- 2019Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Taskscited by 80
- 2024Learning Visuotactile Skills with Two Multifingered Handscited by 71
- 20243D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensingcited by 67
- 2022See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulationcited by 61
- 2025RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learningcited by 47
- 2025Novel Demonstration Generation with Gaussian Splatting Enables Robust One-Shot Manipulationcited by 45
- 2022Visuo-Tactile Transformers for Manipulationcited by 43
- 2021GelSlim3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Fingercited by 39
- 2024Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learningcited by 36
- 2024Multimodal Visual-Tactile Representation Learning through Self-Supervised Contrastive Pre-Trainingcited by 30
- 2023RL-ViGen: A Reinforcement Learning Benchmark for Visual Generalizationcited by 27
- 2023The Power of the Senses: Generalizable Manipulation from Vision and Touch through Masked Multimodal Learningcited by 24
- 2023Uni-O4: Unifying Online and Offline Deep Reinforcement Learning with Multi-Step On-Policy Optimizationcited by 23
- 2023Dextrous Tactile In-Hand Manipulation Using a Modular Reinforcement Learning Architecturecited by 17
- 2025H3DP: Triply-Hierarchical Diffusion Policy for Visuomotor Learningcited by 13
- 2024Catch It! Learning to Catch in Flight with Mobile Dexterous Handscited by 10
- 2020Variational Dynamic for Self-Supervised Exploration in Deep Reinforcement Learningcited by 9
- 2023Sim-to-Real Model-Based and Model-Free Deep Reinforcement Learning for Tactile Pushingcited by 8
- 2025ConViTac: Aligning Visual-Tactile Fusion with Contrastive Representationscited by 7
Built on by
Citing papers per year: 2026: 3.
3 papers.