# Dual Distribution Estimation for Zero-shot Noisy Test-Time Adaptation with VLMs

arXiv: https://arxiv.org/abs/2606.25758 · HTML: https://mlsanity.com/paper/2606.25758/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 47 of its 68 references that are in the map.
- First posted on arXiv 24 Jun 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 30 of 47)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [Learning Robust Global Representations by Penalizing Local Predictive Power](https://mlsanity.com/paper/1905.13549/) (2019)
- [Towards Stable Test-Time Adaptation in Dynamic Wild World](https://mlsanity.com/paper/2302.12400/) (2023)
- [Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models](https://mlsanity.com/paper/2209.07511/) (2022)
- [Delving into Out-of-Distribution Detection with Vision-Language Representations](https://mlsanity.com/paper/2211.13445/) (2022)
- [FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving](https://mlsanity.com/paper/2505.17685/) (2025)
- [Efficient Test-Time Adaptation of Vision-Language Models](https://mlsanity.com/paper/2403.18293/) (2024)
- [The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization](https://mlsanity.com/paper/2112.00463/) (2021)
- [DELTA: degradation-free fully test-time adaptation](https://mlsanity.com/paper/2301.13018/) (2023)
- [Negative Label Guided OOD Detection with Pretrained Vision-Language Models](https://mlsanity.com/paper/2403.20078/) (2024)
- [Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Models](https://mlsanity.com/paper/2403.17589/) (2024)
- [Dual Prototype Evolving for Test-Time Generalization of Vision-Language Models](https://mlsanity.com/paper/2410.12790/) (2024)
- [On the Robustness of Open-World Test-Time Training: Self-Training with Dynamic Prototype Expansion](https://mlsanity.com/paper/2308.09942/) (2023)
- [Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Models](https://mlsanity.com/paper/2403.12952/) (2024)
- [MedEyes: Learning Dynamic Visual Focus for Medical Progressive Diagnosis](https://mlsanity.com/paper/2511.22018/) (2025)
- [Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advances](https://mlsanity.com/paper/2409.11884/) (2024)
- [Enhancing Zero-Shot Vision Models by Label-Free Prompt Distribution Learning and Bias Correcting](https://mlsanity.com/paper/2410.19294/) (2024)
- [DOTA: Distributional Test-Time Adaptation of Vision-Language Models](https://mlsanity.com/paper/2409.19375/) (2024)
- [Bayesian Test-Time Adaptation for Vision-Language Models](https://mlsanity.com/paper/2503.09248/) (2025)
- [LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Models](https://mlsanity.com/paper/2407.08966/) (2024)
- [Thinking with Blueprints: Assisting Vision-Language Models in Spatial Reasoning via Structured Object Representation](https://mlsanity.com/paper/2601.01984/) (2026)
- [OODD: Test-time Out-of-Distribution Detection with Dynamic Dictionary](https://mlsanity.com/paper/2503.10468/) (2025)
- [AdaNeg: Adaptive Negative Proxy Guided OOD Detection with Vision-Language Models](https://mlsanity.com/paper/2410.20149/) (2024)
- [On the Efficacy of Small Self-Supervised Contrastive Models without Distillation Signals](https://mlsanity.com/paper/2107.14762/) (2021)
- [AMMKD: Adaptive Multimodal Multi-teacher Distillation for Lightweight Vision-Language Models](https://mlsanity.com/paper/2509.00039/) (2025)
- [MMT-ARD: Multimodal Multi-Teacher Adversarial Distillation for Robust Vision-Language Models](https://mlsanity.com/paper/2511.17448/) (2025)

## Built on by (most cited first, 2 of 2)

- [MMOOC: A Comprehensive Benchmark for Out-of-Context Evaluation in Multimodal Large Language Models](https://mlsanity.com/paper/2607.27637/) (2026)
- [LightAIR: Lightweight Action Inversion and Riemannian Rectification for Text-based Person Anomaly Search](https://mlsanity.com/paper/2608.09152/) (2026)

## Abstract (arXiv)

While test-time adaptation (TTA) empowers vision-language models to adapt without costly retraining, it remains highly vulnerable to out-of-distribution (OOD) outliers prevalent in real-world applications. This discrepancy motivates Noisy TTA (NTTA), an online task to filter noisy OOD samples on the fly while maximizing in-distribution (ID) classification accuracy. Existing zero-shot NTTA approaches typically rely on test-time discriminative training, leading to overconfident misclassifications and significantly degraded inference efficiency. To address these limitations, we propose a novel framework named Dual Distribution Estimation (DDE), shifting the zero-shot NTTA paradigm from instance-level learning to training-free Gaussian distribution modeling. DDE incorporates two novel modules: Positive Feature Distribution Estimation (PFDE) and Negative Label Distribution Estimation (NLDE). PFDE explicitly models class-wise inclusion and exclusion Gaussian distributions to formulate a calibrated contrastive score, robustly enhancing ID accuracy. In parallel, NLDE improves OOD identification by explicitly modeling the negative label distribution to mine highly discriminative labels, effectively mitigating spurious correlations. Extensive experiments show that on the large-scale ImageNet benchmark, DDE achieves an improvement of 3.70% in harmonic mean accuracy and reduces the FPR95 for OOD detection by 6.20%, while ensuring highly scalable and efficient online inference. Furthermore, DDE is zero-shot and training-free, demonstrating remarkable robustness in data-scarce scenarios. Codes are available at https://github.com/ZhuWenjie98/DDE.