mlsanityData to 14 Oct 2026

Majorization-Guided Test-Time Adaptation for Vision-Language Models under Modality-Specific Shift

arXiv 2604.24602 ↗First posted 27 Apr 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Vision--language models can face asymmetric visual and textual shifts at deployment. These shifts expose a multimodal failure mode in which an unreliable branch remains overconfident, dominates fusion, and causes entropy-based test-time adaptation to sharpen an incorrect prediction. We model this behavior as doubly stochastic posterior mixing and cast adaptation as constrained de-mixing. Majorization-Guided Multimodal Test-Time Adaptation (MG-MTTA) freezes both encoders and updates only a lightweight fusion module. Running-anchor consistency estimates relative branch drift, while cross-modal conflict regulates modality dominance before entropy sharpening. The analysis gives sufficient conditions for entropy reduction to preserve the clean decision and an explicit threshold at which a biased modality reverses the fused ranking. Across visual, textual, and joint shifts, MG-MTTA improves ImageNet top-1 accuracy from 57.97% to 66.51% under textual shift and from 21.68% to 26.27% under joint shift, while reducing wrong-more-confident failures. The largest gains occur under textual and joint shifts, where the two branches differ more in reliability. Project page: https://mg-mtta.github.io/.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 11 of its 29 references that are in the map. First posted on arXiv 27 Apr 2026.

Builds on

11 of 29 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Efficient Test-Time Adaptation of Vision-Language Models27 Mar

Other references · 10

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  3. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  4. 2022Efficient Test-Time Model Adaptation without Forgettingcited by 469
  5. 2023Towards Stable Test-Time Adaptation in Dynamic Wild Worldcited by 415
  6. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 359
  7. 2023Provable Dynamic Fusion for Low-Quality Multimodal Datacited by 82
  8. 2024Multimodal Alignment and Fusion: A Surveycited by 74
  9. 2026ReTrack: Evidence-Driven Dual-Stream Directional Anchor Calibration Network for Composed Video Retrievalcited by 41
  10. 2024textttBATCLIP: Bimodal Online Test-Time Adaptation for CLIPcited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Advancing Reliable Test-Time Adaptation of Vision-Language Models under Visual Variations
  2. 2025Demystifying the Visual Quality Paradox in Multimodal Large Language Models
  3. 2024Frustratingly Easy Test-Time Adaptation of Vision-Language Models
  4. 2026Test-Time Training for Modality Order Consistency in Vision-Language Models
  5. 2025Test-Time Consistency in Vision Language Models
  6. 2025D-TPT: Dimensional Entropy Maximization for Calibrating Test-Time Prompt Tuning in Vision-Language Models
  7. 2026MuRA: Multi-Rank Adaptation for Efficient and Effective Test-Time Vision-Language Generalization
  8. 2024Dual Prototype Evolving for Test-Time Generalization of Vision-Language Models