mlsanityData to 7 Oct 2026

COSMIC: Clique-Oriented Semantic Multi-space Integration for Robust CLIP Test-Time Adaptation

arXiv 2503.23388 ↗First posted 30 Mar 2025cs.CV cs.AI cs.LGcited by 13

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Recent vision-language models (VLMs) face significant challenges in test-time adaptation to novel domains. While cache-based methods show promise by leveraging historical information, they struggle with both caching unreliable feature-label pairs and indiscriminately using single-class information during querying, significantly compromising adaptation accuracy. To address these limitations, we propose COSMIC (Clique-Oriented Semantic Multi-space Integration for CLIP), a robust test-time adaptation framework that enhances adaptability through multi-granular, cross-modal semantic caching and graph-based querying mechanisms. Our framework introduces two key innovations: Dual Semantics Graph (DSG) and Clique Guided Hyper-class (CGH). The Dual Semantics Graph constructs complementary semantic spaces by incorporating textual features, coarse-grained CLIP features, and fine-grained DINOv2 features to capture rich semantic relationships. Building upon these dual graphs, the Clique Guided Hyper-class component leverages structured class relationships to enhance prediction robustness through correlated class selection. Extensive experiments demonstrate COSMIC's superior performance across multiple benchmarks, achieving significant improvements over state-of-the-art methods: 15.81% gain on out-of-distribution tasks and 5.33% on cross-domain generation with CLIP RN-50. Code is available at github.com/hf618/COSMIC.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 43 references that are in the map. First posted on arXiv 30 Mar 2025.

Builds on

31 of 43 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024DOTA: Distributional Test-Time Adaptation of Vision-Language Models28 Sep
  2. 2024Efficient Test-Time Adaptation of Vision-Language Models27 Mar
  3. 2024Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Models26 Mar
  4. 2023Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalization2 Nov
  5. 2023Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning11 Aug
  6. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models15 Sep
  7. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  8. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling6 Nov
  9. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  6. 2013Describing Textures in the Wildcited by 2,110
  7. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  8. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  9. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  10. 2019Natural Adversarial Examplescited by 1,401
  11. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  12. 2022Language-driven Semantic Segmentationcited by 711
10 more
  1. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  2. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  3. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
  4. 2021DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentationcited by 299
  5. 2023Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learnerscited by 152
  6. 2021CLIP4Caption: CLIP for Video Captioncited by 88
  7. 20233D Registration with Maximal Cliquescited by 83
  8. 2023GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graphcited by 69
  9. 2023RM-Depth: Unsupervised Learning of Recurrent Monocular Depth in Dynamic Scenescited by 22
  10. 2024DTBS: Dual-Teacher Bi-directional Self-training for Domain Adaptation in Nighttime Semantic Segmentationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 11.

13 papers.

Sort:

By area:

  1. 2025Test-Time Distillation for Continual Model Adaptationcited by 5
  2. 2026Test-Time Adaptation via Cache Personalization for Facial Expression Recognition in Videoscited by 5
  3. 2026Exposure Bias Can Alleviate Itself via Directional and Frequency Rectification in Flow Matchingcited by 4
  4. 2026Dual-level Adaptation for Multi-Object Tracking: Building Test-Time Calibration from Experience and Intuitioncited by 2
  5. 2026Dance Across Shifts: Forward-Facilitation Continual Test-Time Adaptation through Dynamic Style Bridgingcited by 2
  6. 2025Neural Collapse in Test-Time Adaptationcited by 1
  7. 2026When CLIP Sees More, It Fights Back Harder: Multi-View Guided Adaptive Counterattacks for Test-Time Adversarial Robustnesscited by 1
  8. 2026Beyond Heuristic Prompting: A Concept-Guided Bayesian Framework for Zero-Shot Image Recognitioncited by 0
  9. 2026Training-Only Heterogeneous Image-Patch-Text Graph Supervision for Advancing Few-Shot Learning Adapterscited by 0
  10. 2026Geometry-Preserving Unsupervised Alignment for Heterogeneous Foundation Modelscited by 0
  11. 2026One Stone, Three Birds: Self-adaptive Optimal Transport for Multi-VLM Selection, Adaptation, and Ensemblingcited by 0
  12. 2026Towards Purified Multi-Label Test-Time Adaptation of Vision-Language Modelscited by 0
1 more
  1. 2026Retrieval Geometry Shapes Cache-Based Clip Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024CLIPArTT: Adaptation of CLIP to New Domains at Test Time
  2. 2024Semantic Anchor Transport: Robust Test-Time Adaptation for Vision-Language Models
  3. 2026RADC: Risk-Aware Dual Caching for Vision-Language Test-Time Adaptation
  4. 2025Unleashing the Potential of All Test Samples: Mean-Shift Guided Test-Time Adaptation
  5. 2025QueryAdapter: Rapid Adaptation of Vision-Language Models in Response to Natural Language Queries
  6. 2025SCAP: Transductive Test-Time Adaptation via Supportive Clique-based Attribute Prompting
  7. 2026Sparse MLLM Anchors, Dense Adaptation: Breaking the Self-Referential Loop in Wild Test-Time Adaptation
  8. 2026Ramen: Robust Test-Time Adaptation of Vision-Language Models with Active Sample Selection