mlsanityData to 7 Oct 2026

Test-Time Adaptation for Tactile-Vision-Language Models

arXiv 2602.15873 ↗First posted 31 Jan 2026cs.RO cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Tactile-vision-language (TVL) models are increasingly deployed in real-world robotic and multimodal perception tasks, where test-time distribution shifts are unavoidable. Existing test-time adaptation (TTA) methods provide filtering in unimodal settings but lack explicit treatment of modality-wise reliability under asynchronous cross-modal shifts, leaving them brittle when some modalities become unreliable. We study TTA for TVL models under such shifts and propose a reliability-aware framework that estimates per-modality reliability from prediction uncertainty and perturbation-based responses. This shared reliability signal is used to (i) filter unreliable test samples, (ii) adaptively fuse tactile, visual, and language features, and (iii) regularize test-time optimization with a reliability-guided objective. On the TAG-C benchmark and additional TVL scenarios, our approach consistently outperforms strong TTA baselines, achieving accuracy gains of up to 49.9% under severe modality corruptions, underscoring the importance of explicit modality-wise reliability modeling for robust test-time adaptation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 27 references that are in the map. First posted on arXiv 31 Jan 2026.

Builds on

20 of 27 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Efficient Test-Time Adaptation of Vision-Language Models27 Mar

Other references · 19

  1. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  2. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  3. 2022Continual Test-Time Domain Adaptationcited by 520
  4. 2022Efficient Test-Time Model Adaptation without Forgettingcited by 467
  5. 2023Towards Stable Test-Time Adaptation in Dynamic Wild Worldcited by 409
  6. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 376
  7. 2020DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulationcited by 164
  8. 2024Entropy is not Enough for Test-Time Adaptation: From the Perspective of Disentangled Factorscited by 106
  9. 2018More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touchcited by 100
  10. 2024Binding Touch to Everything: Learning Unified Multimodal Tactile Representationscited by 94
  11. 2022Touch and Go: Learning from Human-Collected Vision and Touchcited by 84
  12. 2022MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentationcited by 76
7 more
  1. 2022See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulationcited by 61
  2. 2018GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Fingercited by 50
  3. 2025AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensorscited by 46
  4. 2021Act the Part: Learning Interaction Strategies for Articulated Object Part Discoverycited by 45
  5. 2019Learning to Identify Object Instances by Touch: Tactile Recognition via Multimodal Matchingcited by 16
  6. 2025Smoothing the Shift: Towards Stable Test-Time Adaptation under Complex Multimodal Noisescited by 9
  7. 2025Attention Bootstrapping for Multi-Modal Test-Time Adaptationcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models
  2. 2026Tactile Modality Fusion for Vision-Language-Action Models
  3. 2026AVT-Fabric: Active Visuo-Tactile Perception via Adaptive Evidence Selection for Efficient Robotic Fabric Comparison ↗
  4. 2026Majorization-Guided Test-Time Adaptation for Vision-Language Models under Modality-Specific Shift
  5. 2026AT-VLA: Adaptive Tactile Injection for Enhanced Feedback Reaction in Vision-Language-Action Models
  6. 2026TAP-VLA: Tactile Annotation Prompting for Vision Language Action Models
  7. 2026ForeTac-VLA: A Forecasting-Based Tactile-Vision-Language-Action Model for Contact-Rich Robotic Manipulation
  8. 2026N0-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens