# Subspace Alignment for Vision-Language Model Test-time Adaptation

arXiv: https://arxiv.org/abs/2601.08139 · HTML: https://mlsanity.com/paper/2601.08139/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 27 of its 106 references that are in the map.
- First posted on arXiv 13 Jan 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 27 of 27)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://mlsanity.com/paper/2205.01917/) (2022)
- [PaLI: A Jointly-Scaled Multilingual Language-Image Model](https://mlsanity.com/paper/2209.06794/) (2022)
- [Tent: Fully Test-time Adaptation by Entropy Minimization](https://mlsanity.com/paper/2006.10726/) (2020)
- [Improving LLM General Preference Alignment via Optimistic Online Mirror Descent](https://mlsanity.com/paper/2502.16852/) (2025)
- [Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs](https://mlsanity.com/paper/2508.14896/) (2025)
- [Visual question answering: from early developments to recent advances -- a survey](https://mlsanity.com/paper/2501.03939/) (2025)
- [Pave Your Own Path: Graph Gradual Domain Adaptation on Fused Gromov-Wasserstein Geodesics](https://mlsanity.com/paper/2505.12709/) (2025)
- [ResMoE: Space-efficient Compression of Mixture of Experts LLMs via Residual Restoration](https://mlsanity.com/paper/2503.06881/) (2025)
- [Subspace Alignment For Domain Adaptation](https://mlsanity.com/paper/1409.5241/) (2014)
- [Matcha: Mitigating Graph Structure Shifts with Test-Time Adaptation](https://mlsanity.com/paper/2410.06976/) (2024)
- [A Collaborative Ensemble Framework for CTR Prediction](https://mlsanity.com/paper/2411.13700/) (2024)
- [MORALISE: A Structured Benchmark for Moral Alignment in Visual Language Models](https://mlsanity.com/paper/2505.14728/) (2025)
- [PLANETALIGN: A Comprehensive Python Library for Benchmarking Network Alignment](https://mlsanity.com/paper/2505.21366/) (2025)
- [Hierarchical LoRA MoE for Efficient CTR Model Scaling](https://mlsanity.com/paper/2510.10432/) (2025)
- [Harnessing Consistency for Robust Test-Time LLM Ensemble](https://mlsanity.com/paper/2510.13855/) (2025)
- [CATS: Mitigating Correlation Shift for Multivariate Time Series Classification](https://mlsanity.com/paper/2504.04283/) (2025)
- [THeGCN: Temporal Heterophilic Graph Convolutional Network](https://mlsanity.com/paper/2412.16435/) (2024)
- [Mint: A Simple Test-Time Adaptation of Vision-Language Models against Common Corruptions](https://mlsanity.com/paper/2510.22127/) (2025)
- [Panda: Test-Time Adaptation with Negative Data Augmentation](https://mlsanity.com/paper/2511.10481/) (2025)
- [SimCE: Simplifying Cross-Entropy Loss for Collaborative Filtering](https://mlsanity.com/paper/2406.16170/) (2024)
- [AdaFuse: Adaptive Ensemble Decoding with Test-Time Scaling for LLMs](https://mlsanity.com/paper/2601.06022/) (2026)
- [ALERT: Zero-shot LLM Jailbreak Detection via Internal Discrepancy Amplification](https://mlsanity.com/paper/2601.03600/) (2026)
- [Noisy Positive-Unlabeled Learning with Self-Training for Speculative Knowledge Graph Reasoning](https://mlsanity.com/paper/2306.07512/) (2023)
- [Complementary Subspace Low-Rank Adaptation of Vision-Language Models for Few-Shot Classification](https://mlsanity.com/paper/2501.15040/) (2025)

## Built on by (most cited first, 5 of 5)

- [Mixture of Sequence: Theme-Aware Mixture-of-Experts for Long-Sequence Recommendation](https://mlsanity.com/paper/2604.20858/) (2026)
- [Graph homophily booster: Reimagining the role of discrete features in heterophilic graph learning](https://mlsanity.com/paper/2602.07256/) (2026)
- [ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuning](https://mlsanity.com/paper/2603.10160/) (2026)
- [AvAtar: Learning to Align via Active Optimal Transport](https://mlsanity.com/paper/2605.24395/) (2026)
- [Ramen: Robust Test-Time Adaptation of Vision-Language Models with Active Sample Selection](https://mlsanity.com/paper/2604.21728/) (2026)

## Abstract (arXiv)

Vision-language models (VLMs), despite their extraordinary zero-shot capabilities, are vulnerable to distribution shifts. Test-time adaptation (TTA) emerges as a predominant strategy to adapt VLMs to unlabeled test data on the fly. However, existing TTA methods heavily rely on zero-shot predictions as pseudo-labels for self-training, which can be unreliable under distribution shifts and misguide adaptation due to two fundamental limitations.First (Modality Gap), distribution shifts induce gaps between visual and textual modalities, making cross-modal relations inaccurate. Second (Visual Nuisance), visual embeddings encode rich but task-irrelevant noise that often overwhelms task-specific semantics under distribution shifts. To address these limitations, we propose SubTTA, which aligns the semantic subspaces of both modalities to enhance zero-shot predictions to better guide the TTA process. To bridge the modality gap, SubTTA extracts the principal subspaces of both modalities and aligns the visual manifold to the textual semantic anchor by minimizing their chordal distance. To eliminate visual nuisance, SubTTA projects the aligned visual features onto the task-specific textual subspace, which filters out task-irrelevant noise by constraining visual embeddings within the valid semantic span, and standard TTA is further performed on the purified space to refine the decision boundaries. Extensive experiments on various benchmarks and VLM architectures demonstrate the effectiveness of SubTTA, yielding an average improvement of 2.24% over state-of-the-art TTA methods. Our code is available at https://github.com/zhichenz98/SubTTA_EMNLP26.