mlsanityData to 14 Oct 2026

ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptation

arXiv 2308.03793 ↗First posted 4 Aug 2023cs.CV cs.LGcited by 35

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Large-scale Pre-Training Vision-Language Model such as CLIP has demonstrated outstanding performance in zero-shot classification, e.g. achieving 76.3% top-1 accuracy on ImageNet without seeing any example, which leads to potential benefits to many tasks that have no labeled data. However, while applying CLIP to a downstream target domain, the presence of visual and text domain gaps and cross-modality misalignment can greatly impact the model performance. To address such challenges, we propose ReCLIP, the first source-free domain adaptation method for vision-language models, which does not require any source data or target labeled data. ReCLIP first learns a projection space to mitigate the misaligned visual-text embeddings and learns pseudo labels, and then deploys cross-modality self-training with the pseudo labels, to update visual and text encoders, refine labels and reduce domain gaps and misalignments iteratively. With extensive experiments, we demonstrate ReCLIP reduces the average error rate of CLIP from 30.17% to 25.06% on 22 image classification benchmarks. Code available at https://github.com/michiganleon/ReCLIP_WACV.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 61 references that are in the map. First posted on arXiv 4 Aug 2023.

Builds on

34 of 61 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2016Layer Normalizationcited by 5,669
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  8. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  10. 2013Describing Textures in the Wildcited by 2,125
  11. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  12. 2022Visual Prompt Tuningcited by 1,681
22 more
  1. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  2. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  3. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,188
  4. 2020Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptationcited by 848
  5. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 797
  6. 2018Deep Visual Domain Adaptation: A Surveycited by 635
  7. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 561
  8. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 490
  9. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 436
  10. 2018Rotation Equivariant CNNs for Digital Pathologycited by 422
  11. 2019Contrastive Adaptation Network for Unsupervised Domain Adaptationcited by 407
  12. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  13. 2016AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classificationcited by 347
  14. 2019Semi-supervised Domain Adaptation via Minimax Entropycited by 335
  15. 2019Label Propagation for Deep Semi-supervised Learningcited by 289
  16. 2017Domain Adaptation for Visual Applications: A Comprehensive Surveycited by 128
  17. 2020FixBi: Bridging Domain Spaces for Unsupervised Domain Adaptationcited by 124
  18. 2019Unsupervised Domain Adaptation through Self-Supervisioncited by 116
  19. 2022Attracting and Dispersing: A Simple Approach for Source-free Domain Adaptationcited by 94
  20. 2021MetaAlign: Coordinating Domain Alignment and Classification for Unsupervised Domain Adaptationcited by 54
  21. 2023POUF: Prompt-oriented unsupervised fine-tuning for large pre-trained modelscited by 29
  22. 2021Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptationcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 15; 2025: 15; 2026: 4.

35 papers.

Sort:

By area:

  1. 2024Label Propagation for Zero-shot Classification with Vision-Language Modelscited by 28
  2. 2024Mitigate the Gap: Investigating Approaches for Improving Cross-Modal Alignment in CLIPcited by 23
  3. 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Modelscited by 23
  4. 2025LPOSS: Label Propagation Over Patches and Pixels for Open-vocabulary Semantic Segmentationcited by 21
  5. 2024Semantic-Aligned Adversarial Evolution Triangle for High-Transferability Vision-Language Attackcited by 18
  6. 2024Source-Free Domain Adaptation Guided by Vision and Vision-Language Pre-Trainingcited by 15
  7. 2025CLIP-Powered Domain Generalization and Domain Adaptation: A Comprehensive Surveycited by 15
  8. 2024A Lost Opportunity for Vision-Language Models: A Comparative Study of Online Test-Time Adaptation for Vision-Language Modelscited by 11
  9. 2024DENOISER: Rethinking the Robustness for Open-Vocabulary Action Recognitioncited by 8
  10. 2024Efficient and Context-Aware Label Propagation for Zero-/Few-Shot Training-Free Adaptation of Vision-Language Modelcited by 8
  11. 2025Details Matter for Indoor Open-vocabulary 3D Instance Segmentationcited by 8
  12. 2025Generalizing vision-language models to novel domains: A comprehensive surveycited by 7
23 more
  1. 2025CoPT: Unsupervised Domain Adaptive Segmentation using Domain-Agnostic Text Embeddingscited by 7
  2. 2024Tuning-free Universally-Supervised Semantic Segmentationcited by 6
  3. 2025Simulations of Common Unsupervised Domain Adaptation Algorithms for Image Classificationcited by 6
  4. 2024DPA: Dual Prototypes Alignment for Unsupervised Adaptation of Vision-Language Modelscited by 5
  5. 2025GenZSL: Generative Zero-Shot Learning Via Inductive Variational Autoencodercited by 5
  6. 2024Language-Guided Instance-Aware Domain-Adaptive Panoptic Segmentationcited by 4
  7. 2024Industrial Language-Image Dataset (ILID): Adapting Vision Foundation Models for Industrial Settingscited by 4
  8. 2025FLOSS: Free Lunch in Open-vocabulary Semantic Segmentationcited by 4
  9. 2024Empowering Source-Free Domain Adaptation via MLLM-Guided Reliability-Based Curriculum Learningcited by 3
  10. 2024LatteCLIP: Unsupervised CLIP Fine-Tuning via LMM-Synthetic Textscited by 3
  11. 2025Federated Domain Generalization with Domain-specific Soft Prompts Generationcited by 3
  12. 2024BaFTA: Backprop-Free Test-Time Adaptation For Zero-Shot Vision-Language Modelscited by 2
  13. 2025ViLAaD: Enhancing "Attracting and Dispersing'' Source-Free Domain Adaptation with Vision-and-Language Modelcited by 2
  14. 2024LARE: Latent Augmentation using Regional Embedding with Vision-Language Modelcited by 1
  15. 2025Towards Fine-Grained Adaptation of CLIP via a Self-Trained Alignment Scorecited by 1
  16. 2023Noise-Tolerant Few-Shot Unsupervised Adapter for Vision-Language Modelscited by 0
  17. 2025Exploiting Vision Language Model for Training-Free 3D Point Cloud OOD Detection via Graph Score Propagationcited by 0
  18. 2025Prototype-Guided Pseudo-Labeling with Neighborhood-Aware Consistency for Unsupervised Adaptationcited by 0
  19. 2025microCLIP: Unsupervised CLIP Adaptation via Coarse-Fine Token Fusion for Fine-Grained Image Classificationcited by 0
  20. 2026Zero-Shot Distracted Driver Detection via Vision Language Models with Double Decouplingcited by 0
  21. 2026On the Cone Effect and Modality Gap in Medical Vision-Language Embeddingscited by 0
  22. 2026Dual-Stream Semantic Guidance with Prototype Anchor Calibration for Source-Fully-Free Adaptation of Vision-Language Modelscited by 0
  23. 2026Safeguarding Mutual Correction in Source-Free Domain Adaptation via Cut Statisticscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Rethinking Domain Adaptation and Generalization in the Era of CLIP
  2. 2022UniCLIP: Unified Framework for Contrastive Language-Image Pre-training
  3. 2026Label Shift Aware Adaptation for Online Zero-shot Learning with Contrastive Language-Image Pre-Training (CLIP)
  4. 2025Post-pre-training for Modality Alignment in Vision-Language Foundation Models
  5. 2022SuS-X: Training-Free Name-Only Transfer of Vision-Language Models
  6. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification
  7. 2023Getting More Juice Out of Your Data: Hard Pair Refinement Enhances Visual-Language Models Without Extra Data
  8. 2024CLIP's Visual Embedding Projector is a Few-shot Cornucopia