mlsanityData to 14 Oct 2026

What Does Prompt Learning Change? -A Natural-Language Concept Analysis of Vision-Language Models

arXiv 2608.24142 ↗First posted 25 Aug 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Prompt learning adapts vision-language models such as CLIP by optimizing continuous prompt vectors, but the learned prompts are difficult to interpret in natural language. We present PromptSpLiCE, a post-hoc method that expresses each class-conditioned text embedding as a sparse combination of concepts from a fixed natural-language dictionary. Using the same dictionary before and after prompt learning allows us to compare changes in their concept profiles. We evaluate PromptSpLiCE on CoOp, a representative prompt-learning method, across 11 image-classification datasets. The concept profiles change substantially: on average, only 1.6 of the initial top-10 concepts remain in the top 10 after learning. Across datasets, profile change is positively associated with accuracy gain. We also derive a local gradient expression that provides geometric intuition for why image-aligned concept directions distinct from the current prompt can have greater loss sensitivity.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 25 of its 36 references that are in the map. First posted on arXiv 25 Aug 2026.

Builds on

25 of 36 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Interpretable Zero-Shot Learning with Locally-Aligned Vision-Language Model30 Jun
  2. 2024TextRefiner: Internal Visual Feature as Efficient Refiner for Vision-Language Models Prompt Tuning11 Dec
  3. 2023Visual-Language Prompt Tuning with Knowledge-guided Context Optimization23 Mar
  4. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  5. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  5. 2013Describing Textures in the Wildcited by 2,125
  6. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  7. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  8. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  9. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  10. 2022MaPLe: Multi-modal Prompt Learningcited by 687
  11. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 561
  12. 2023DeCap: Decoding CLIP Latents for Zero-Shot Captioning via Text-Only Trainingcited by 84
8 more
  1. 2025Sparse Autoencoders Learn Monosemantic Features in Vision-Language Modelscited by 71
  2. 2024Sparse autoencoders reveal selective remapping of visual concepts during adaptationcited by 68
  3. 2025Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignmentcited by 62
  4. 2022LASP: Text-to-Text Optimization for Language-Aware Soft Prompting of Vision & Language Modelscited by 59
  5. 2024XCoOp: Explainable Prompt Learning for Computer-Aided Diagnosis via Concept-guided Context Optimizationcited by 20
  6. 2025SAUCE: Selective Concept Unlearning in Vision-Language Models with Sparse Autoencoderscited by 13
  7. 2024IntCoOp: Interpretability-Aware Vision-Language Prompt Tuningcited by 5
  8. 2026SEM: Sparse Embedding Modulation for Post-Hoc Debiasing of Vision-Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Aggregate-and-Adapt Natural Language Prompts for Downstream Generalization of CLIP
  2. 2025Decoupling Augmentation Bias in Prompt Learning for Vision-Language Models
  3. 2024Learning to Prompt with Text Only Supervision for Vision-Language Models
  4. 2024Revisiting Prompt Pretraining of Vision-Language Models
  5. 2023DPL: Decoupled Prompt Learning for Vision-Language Models
  6. 2024Tuning Vision-Language Models with Candidate Labels by Prompt Alignment
  7. 2022Learning Domain Invariant Prompt for Vision-Language Models
  8. 2023CoPL: Contextual Prompt Learning for Vision-Language Understanding