mlsanityData to 7 Oct 2026

Domain-Controlled Prompt Learning

arXiv 2310.07730 ↗First posted 30 Sep 2023cs.CV eess.IVcited by 22

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Large pre-trained vision-language models, such as CLIP, have shown remarkable generalization capabilities across various tasks when appropriate text prompts are provided. However, adapting these models to specific domains, like remote sensing images (RSIs), medical images, etc, remains unexplored and challenging. Existing prompt learning methods often lack domain-awareness or domain-transfer mechanisms, leading to suboptimal performance due to the misinterpretation of specific images in natural image patterns. To tackle this dilemma, we proposed a Domain-Controlled Prompt Learning for the specific domains. Specifically, the large-scale specific domain foundation model (LSDM) is first introduced to provide essential specific domain knowledge. Using lightweight neural networks, we transfer this knowledge into domain biases, which control both the visual and language branches to obtain domain-adaptive prompts in a directly incorporating manner. Simultaneously, to overcome the existing overfitting challenge, we propose a novel noisy-adding strategy, without extra trainable parameters, to help the model escape the suboptimal solution in a global domain oscillation manner. Experimental results show our method achieves state-of-the-art performance in specific domain image recognition datasets. Our code is available at https://github.com/caoql98/DCPL.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 41 of its 57 references that are in the map. First posted on arXiv 30 Sep 2023.

Builds on

41 of 57 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  2. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling6 Nov
  3. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapters9 Oct
  4. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  9. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  10. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  11. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  12. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
25 more
  1. 2015Fast R-CNNcited by 2,930
  2. 2021Learning to Prompt for Continual Learningcited by 938
  3. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  4. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
  5. 2022Language-driven Semantic Segmentationcited by 711
  6. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  7. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  8. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  9. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 530
  10. 2021Image Segmentation Using Text and Image Promptscited by 509
  11. 2016AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classificationcited by 343
  12. 2013Zero-Shot Learning Through Cross-Modal Transfercited by 237
  13. 2021ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Biascited by 216
  14. 2017Exploring Models and Data for Remote Sensing Image Caption Generationcited by 195
  15. 2015Learning Visual Features from Large Weakly Supervised Datacited by 165
  16. 2022Advancing Plain Vision Transformer Towards Remote Sensing Foundation Modelcited by 149
  17. 2022Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detectioncited by 145
  18. 2022PromptDet: Towards Open-vocabulary Detection using Uncurated Imagescited by 133
  19. 2022Domain Adaptation via Prompt Learningcited by 128
  20. 2017PatternNet: A Benchmark Dataset for Performance Evaluation of Remote Sensing Image Retrievalcited by 96
  21. 2022CLIP-Art: Contrastive Pre-training for Fine-Grained Art Classificationcited by 65
  22. 2015Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptionscited by 64
  23. 2020MLRSNet: A Multi-label High Spatial Resolution Remote Sensing Dataset for Semantic Scene Understandingcited by 59
  24. 2018FRAGE: Frequency-Agnostic Word Representationcited by 47
  25. 2017Self-supervised learning of visual features through embedding images into text topic spacescited by 43

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 5; 2025: 12; 2026: 3.

22 papers.

Sort:

By area:

  1. 2023CLIP in Medical Imaging: A Surveycited by 40
  2. 2024Advancing Tool-Augmented Large Language Models: Integrating Insights from Errors in Inference Treescited by 38
  3. 2024BiomedCoOp: Learning to Prompt for Biomedical Vision-Language Modelscited by 28
  4. 2024Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptationcited by 25
  5. 2023Domain Prompt Learning with Quaternion Networkscited by 17
  6. 2024One is Plenty: A Polymorphic Feature Interpreter for Immutable Heterogeneous Collaborative Perceptioncited by 15
  7. 2025Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Reviewcited by 9
  8. 2025CoPT: Unsupervised Domain Adaptive Segmentation using Domain-Agnostic Text Embeddingscited by 7
  9. 2025TextSAM-EUS: Text Prompt Learning for SAM to Accurately Segment Pancreatic Tumor in Endoscopic Ultrasoundcited by 5
  10. 2025Medical Knowledge Intervention Prompt Tuning for Medical Image Classificationcited by 5
  11. 2025UPRE: Zero-Shot Domain Adaptation for Object Detection via Unified Prompt and Representation Enhancementcited by 4
  12. 2025Foundation Models for Anomaly Detection: Vision and Challengescited by 3
10 more
  1. 2025MGPATH: Vision-Language Model with Multi-Granular Prompt Learning for Few-Shot WSI Classificationcited by 3
  2. 2025Generalizable Prompt Learning of CLIP: A Brief Overviewcited by 3
  3. 2025CLIP-SVD: Efficient and Interpretable Vision-Language Adaptation via Singular Valuescited by 3
  4. 2026MedCLIPSeg: Probabilistic Vision-Language Adaptation for Data-Efficient and Generalizable Medical Image Segmentationcited by 3
  5. 2024Advancing Prompt Learning through an External Layercited by 2
  6. 2025FrogDogNet: Fourier frequency Retained visual prompt Output Guidance for Domain Generalization of CLIP in Remote Sensingcited by 2
  7. 2025SciTopic: Enhancing Topic Discovery in Scientific Literature through Advanced LLMcited by 1
  8. 2025Brain Imaging Foundation Models, Are We There Yet? A Systematic Review of Foundation Models for Brain Imaging and Biomedical Researchcited by 0
  9. 2026Multi-View Synergistic Learning with Vision-Language Adaption for Low-Resource Biomedical Image Classificationcited by 0
  10. 2026PEARL: A Lightweight Prompt-based Feature Interpreter Framework for Real-Time, Anonymous, and Heterogeneous Collaborative Perceptioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Learning Domain Invariant Prompt for Vision-Language Models
  2. 2024In the Era of Prompt Learning with Vision-Language Models
  3. 2024Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation
  4. 2025Latent Domain Prompt Learning for Vision-Language Models
  5. 2023Learning to Prompt Your Domain for Vision-Language Models
  6. 2024Enhancing Domain Adaptation through Prompt Gradient Alignment
  7. 2024Transitive Vision-Language Prompt Learning for Domain Generalization
  8. 2026Domain-Invariant Prompt Learning for Vision-Language Models