mlsanityData to 7 Oct 2026

CLIPood: Generalizing CLIP to Out-of-Distributions

arXiv 2302.00864 ↗First posted 2 Feb 2023cs.LG cs.CVcited by 96

Not on the map. Why?

Cited by 37 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Out-of-distribution (OOD) generalization, where the model needs to handle distribution shifts from training, is a major challenge of machine learning. Contrastive language-image pre-training (CLIP) models have shown impressive zero-shot ability, but the further adaptation of CLIP on downstream tasks undesirably degrades OOD performances. This paper aims at generalizing CLIP to out-of-distribution test data on downstream tasks. We propose CLIPood, a fine-tuning method that can adapt CLIP models to OOD situations where both domain shifts and open classes may occur on the unseen test data. To exploit the semantic relations between classes from the text modality, CLIPood introduces a new training objective, margin metric softmax (MMS), with class adaptive margins for fine-tuning. To incorporate both pre-trained zero-shot model and fine-tuned task-adaptive model, CLIPood leverages a new optimization strategy, Beta moving average (BMA), to maintain a temporal ensemble weighted by Beta distribution. Experiments on diverse datasets with different OOD scenarios show that CLIPood consistently outperforms existing generalization techniques.

Key facts. Cited by 96 papers in the map (data through 7 Oct 2026). Builds on 53 of its 71 references that are in the map. First posted on arXiv 2 Feb 2023.

Builds on

53 of 71 references.

References · 53

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  6. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  7. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  8. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  9. 2013Describing Textures in the Wildcited by 2,110
  10. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  11. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  12. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
38 more
  1. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  2. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  3. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  4. 2019Natural Adversarial Examplescited by 1,401
  5. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
  6. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  7. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  8. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,181
  9. 2020In Search of Lost Domain Generalizationcited by 1,042
  10. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  11. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  12. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  13. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  14. 2021Robust fine-tuning of zero-shot modelscited by 785
  15. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  16. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  17. 2018Recognition in Terra Incognitacited by 655
  18. 2013Domain Generalization via Invariant Feature Representationcited by 622
  19. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  20. 2018Do Better ImageNet Models Transfer Better?cited by 560
  21. 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 452
  22. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  23. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 449
  24. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 434
  25. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  26. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 429
  27. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  28. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Modelscited by 379
  29. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 248
  30. 2013Zero-Shot Learning Through Cross-Modal Transfercited by 237
  31. 2022Prompt Distribution Learningcited by 235
  32. 2021Combined Scaling for Zero-shot Transfer Learningcited by 182
  33. 2022Finetune like you pretrain: Improved finetuning of zero-shot vision modelscited by 149
  34. 2020Efficient Domain Generalization via Common-Specific Low-Rank Decompositioncited by 134
  35. 2022Domain Generalization by Mutual-Information Regularization with Pre-trained Modelscited by 133
  36. 2019Feature-Critic Networks for Heterogeneous Domain Generalizationcited by 130
  37. 2020Learning to Balance Specificity and Invariance for In and Out of Domain Generalizationcited by 128
  38. 2022CyCLIP: Cyclic Contrastive Language-Image Pretrainingcited by 127

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 11; 2024: 46; 2025: 26; 2026: 13.

96 papers.

Sort:

By area:

  1. 2024Explore the Potential of CLIP for Training-Free Open Vocabulary Semantic Segmentationcited by 61
  2. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Timecited by 57
  3. 2023Leveraging Vision-Language Models for Improving Domain Generalization in Image Classificationcited by 41
  4. 2024Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Surveycited by 36
  5. 2023Towards Calibrated Robust Fine-Tuning of Vision-Language Modelscited by 35
  6. 2024DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learningcited by 32
  7. 2024GalLoP: Learning Global and Local Prompts for Vision-Language Modelscited by 29
  8. 2024PathGen-1.6M: 1.6 Million Pathology Image-text Pairs Generation through Multi-agent Collaborationcited by 28
  9. 2023A Simple Recipe for Language-guided Domain Generalized Segmentationcited by 27
  10. 2023DiffCLIP: Leveraging Stable Diffusion for Language Grounded 3D Classificationcited by 23
  11. 2024Adapting to Distribution Shift by Visual Domain Prompt Generationcited by 23
  12. 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Modelscited by 23
38 more
  1. 2024A Practitioner's Guide to Continual Multimodal Pretrainingcited by 22
  2. 2023Building an Open-Vocabulary Video CLIP Model with Better Architectures, Optimization and Datacited by 21
  3. 2024Robust Domain Generalization for Multi-modal Object Recognitioncited by 18
  4. 2024Overcoming the Pitfalls of Vision-Language Model Finetuning for OOD Generalizationcited by 17
  5. 2024Vision-Language Models are Strong Noisy Label Detectorscited by 16
  6. 2024PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalizationcited by 15
  7. 2025CLIP-Powered Domain Generalization and Domain Adaptation: A Comprehensive Surveycited by 15
  8. 2024Learn to Preserve and Diversify: Parameter-Efficient Group with Orthogonal Regularization for Domain Generalizationcited by 14
  9. 2024Local-Prompt: Extensible Local Prompts for Few-Shot Out-of-Distribution Detectioncited by 14
  10. 2024LanDA: Language-Guided Multi-Source Domain Adaptationcited by 13
  11. 2024CLIPScope: Enhancing Zero-Shot OOD Detection with Bayesian Scoringcited by 13
  12. 2024QT-DoG: Quantization-aware Training for Domain Generalizationcited by 13
  13. 2024Domain-Inspired Sharpness-Aware Minimization Under Domain Shiftscited by 12
  14. 2024SAFT: Towards Out-of-Distribution Generalization in Fine-Tuningcited by 12
  15. 2023Diverse Target and Contribution Scheduling for Domain Generalizationcited by 10
  16. 2024FodFoM: Fake Outlier Data by Foundation Models Creates Stronger Visual Out-of-Distribution Detectorcited by 10
  17. 2024Is Large-Scale Pretraining the Secret to Good Domain Generalization?cited by 9
  18. 2023Robust Fine-Tuning of Vision-Language Models for Domain Generalizationcited by 7
  19. 2024Efficient and Versatile Robust Fine-Tuning of Zero-shot Modelscited by 7
  20. 2025LoRA-TTT: Low-Rank Test-Time Training for Vision-Language Modelscited by 7
  21. 2025Learning to Generalize without Bias for Open-Vocabulary Action Recognitioncited by 7
  22. 2023Improving CLIP Robustness with Knowledge Distillation and Self-Trainingcited by 6
  23. 2023Descriptor and Word Soups: Overcoming the Parameter Efficiency Accuracy Tradeoff for Out-of-Distribution Few-shot Learningcited by 6
  24. 2024CLIP-driven Outliers Synthesis for few-shot OOD detectioncited by 6
  25. 2024HGL: Hierarchical Geometry Learning for Test-time Adaptation in 3D Point Cloud Segmentationcited by 6
  26. 2025OSLoPrompt: Bridging Low-Supervision Challenges and Open-Set Domain Generalization in CLIPcited by 6
  27. 2025Generalizing vision-language models to novel domains: A comprehensive surveycited by 6
  28. 2023Towards Unified and Effective Domain Generalizationcited by 5
  29. 2024Step-wise Distribution Alignment Guided Style Prompt Tuning for Source-free Cross-domain Few-shot Learningcited by 5
  30. 2025Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directionscited by 5
  31. 2025DiMPLe -- Disentangled Multi-Modal Prompt Learning: Enhancing Out-Of-Distribution Alignment with Invariant and Spurious Feature Separationcited by 5
  32. 2026Test-Time Adaptation via Cache Personalization for Facial Expression Recognition in Videoscited by 5
  33. 2024Beyond Finite Data: Towards Data-free Out-of-distribution Generalization via Extrapolationcited by 4
  34. 2024In-context Prompt Learning for Test-time Vision Recognition with Frozen Vision-language Modelcited by 4
  35. 2024Adapting Large Multimodal Models to Distribution Shifts: The Role of In-Context Learningcited by 4
  36. 2024Aggregate-and-Adapt Natural Language Prompts for Downstream Generalization of CLIPcited by 4
  37. 2024Aligning Visual Contrastive learning models via Preference Optimizationcited by 4
  38. 2026Is the Modality Gap a Bug or a Feature? A Robustness Perspectivecited by 4

50 most cited of 96 All 96 (JSON)