Wake up for Touch! Mask-isolated Tactile Alignment Learning in MLLMs
Not on the map. Why?
Abstract (arXiv)
Touch supplies the physical grounding needed to perceive intrinsic material properties, such as friction and compliance, that vision alone often cannot resolve. Recent efforts for equipping multimodal LLMs with this tactile sense, however, expose a zero-sum trade-off: the limited parameter budget of compact models forces a choice between acquiring the new sensory modality and preserving the established vision-language reasoning. We present Splash, a mask-isolated tactile alignment learning framework for MLLMs. Splash quantifies the significance of each pretrained parameter, and partitions the parameter space into a dormant and critical subspace. While the frozen critical subspace acts as a stable anchor to safeguard general visual knowledge, Splash updates the isolated dormant subspace to internalize tactile alignment towards LLMs. This selective, non-destructive expansion effectively prevents catastrophic forgetting and ensures non-destructive modality expansion. Extensive experiments show that Splash effectively achieves tactile reasoning without additional inference overhead in the LLM part, demonstrating state-of-the-art performance on visuo-tactile benchmarks, including SSVTP, TVL, and TacQuad, while preserving its original general-purpose capabilities.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 50 of its 56 references that are in the map. First posted on arXiv 1 Jul 2026.
Builds on
50 of 56 references.
References · 50
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2023Visual Instruction Tuningcited by 7,869
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2024GPT-4o System Cardcited by 4,588
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
38 more
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
- 2016Learning without Forgettingcited by 2,496
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
- 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
- 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
- 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
- 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
- 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
- 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
- 2019Rigging the Lottery: Making All Tickets Winnerscited by 495
- 2024The Unreasonable Ineffectiveness of the Deeper Layerscited by 226
- 2024MobileVLM V2: Faster and Stronger Baseline for Vision Language Modelcited by 174
- 2023Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsitycited by 165
- 2023Investigating the Catastrophic Forgetting in Multimodal Large Language Modelscited by 119
- 2018More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touchcited by 102
- 2024Binding Touch to Everything: Learning Unified Multimodal Tactile Representationscited by 95
- 2022Touch and Go: Learning from Human-Collected Vision and Touchcited by 87
- 2024A Touch, Vision, and Language Dataset for Multimodal Alignmentcited by 69
- 2025AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensorscited by 48
- 2022Self-Supervised Visuo-Tactile Pretraining to Locate and Follow Garment Featurescited by 38
- 2025ViTacFormer: Learning Cross-Modal Representation for Visuo-Tactile Dexterous Manipulationcited by 38
- 2023ViT-Lens: Towards Omni-modal Representationscited by 29
- 2024Touch100k: A Large-Scale Touch-Language-Vision Dataset for Touch-Centric Multimodal Representationcited by 27
- 2017Connecting Look and Feel: Associating the visual and tactile properties of physical materialscited by 22
- 2025Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruningcited by 13
- 2026AnyTouch 2: General Optical Tactile Representation Learning For Dynamic Tactile Perceptioncited by 12
- 2025SparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsitycited by 9
- 2025Lite VLA: Efficient Vision-Language-Action Control on CPU-Bound Edge Robotscited by 9
- 2024Tactile-based force estimation for interaction control with robot fingerscited by 2
- 2021Probabilistic Discriminative Models Address the Tactile Perceptual Aliasing Problemcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Touch-R1: Reinforcing Touch Reasoning in MLLMs
- 2025ViTaPEs: Visuotactile Position Encodings for Cross-Modal Alignment in Multimodal Transformers
- 2026TacMamba: A Tactile History Compression Adapter Bridging Fast Reflexes and Slow VLA Reasoning
- 2026TAP-VLA: Tactile Annotation Prompting for Vision Language Action Models
- 2026TaF-VLA: Tactile-Force Alignment in Vision-Language-Action Models for Force-aware Manipulation
- 2026τ: Learning Touch-Augmented Vision-Language-Action Models from Future Visual Supervision
- 2026AVT-Fabric: Active Visuo-Tactile Perception via Adaptive Evidence Selection for Efficient Robotic Fabric Comparison ↗
- 2025Universal Visuo-Tactile Video Understanding for Embodied Interaction