# ProCAP: Probabilistic Cross-Attentive Prompt Learning for Vision-Language Models

arXiv: https://arxiv.org/abs/2609.30434 · HTML: https://mlsanity.com/paper/2609.30434/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 30 of its 36 references that are in the map.
- First posted on arXiv 24 Sep 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [Learning Robust Global Representations by Penalizing Local Predictive Power](https://mlsanity.com/paper/1905.13549/) (2019)
- [MaPLe: Multi-modal Prompt Learning](https://mlsanity.com/paper/2210.03117/) (2022)
- [Visual-Language Prompt Tuning with Knowledge-guided Context Optimization](https://mlsanity.com/paper/2303.13283/) (2023)
- [Self-regulating Prompts: Foundational Model Adaptation without Forgetting](https://mlsanity.com/paper/2307.06948/) (2023)
- [Eigen-CAM: Class Activation Map using Principal Components](https://mlsanity.com/paper/2008.00299/) (2020)
- [TCP:Textual-based Class-aware Prompt tuning for Visual-Language Model](https://mlsanity.com/paper/2311.18231/) (2023)
- [Consistency-guided Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2306.01195/) (2023)
- [DePT: Decoupled Prompt Tuning](https://mlsanity.com/paper/2309.07439/) (2023)
- [MMRL: Multi-Modal Representation Learning for Vision-Language Models](https://mlsanity.com/paper/2503.08497/) (2025)
- [Cascade Prompt Learning for Vision-Language Model Adaptation](https://mlsanity.com/paper/2409.17805/) (2024)
- [Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation](https://mlsanity.com/paper/2407.03056/) (2024)
- [DPC: Dual-Prompt Collaboration for Tuning Vision-Language Models](https://mlsanity.com/paper/2503.13443/) (2025)
- [Rethinking Few-Shot Adaptation of Vision-Language Models in Two Stages](https://mlsanity.com/paper/2503.11609/) (2025)
- [AAPL: Adding Attributes to Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2404.16804/) (2024)
- [IPO: Interpretable Prompt Optimization for Vision-Language Models](https://mlsanity.com/paper/2410.15397/) (2024)
- [MuAP: Multi-step Adaptive Prompt Learning for Vision-Language Model with Missing Modality](https://mlsanity.com/paper/2409.04693/) (2024)
- [Skip Tuning: Pre-trained Vision-Language Models are Effective and Efficient Adapters Themselves](https://mlsanity.com/paper/2412.11509/) (2024)
- [Style-Pro: Style-Guided Prompt Learning for Generalizable Vision-Language Models](https://mlsanity.com/paper/2411.16018/) (2024)
- [A Similarity Paradigm Through Textual Regularization Without Forgetting](https://mlsanity.com/paper/2502.14376/) (2025)
- [DiMPLe -- Disentangled Multi-Modal Prompt Learning: Enhancing Out-Of-Distribution Alignment with Invariant and Spurious Feature Separation](https://mlsanity.com/paper/2506.21237/) (2025)
- [MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptation](https://mlsanity.com/paper/2602.21397/) (2026)
- [In-context Prompt Learning for Test-time Vision Recognition with Frozen Vision-language Model](https://mlsanity.com/paper/2403.06126/) (2024)
- [Mixture of Prompt Learning for Vision Language Models](https://mlsanity.com/paper/2409.12011/) (2024)
- [Multi-modal Mutual-Guidance Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2507.08410/) (2025)
- [Adaptive Multi-Modality Prompt Learning](https://mlsanity.com/paper/2312.00823/) (2023)

## Abstract (arXiv)

Pre-trained vision-language models such as CLIP can recognize new categories via prompting, but they often struggle when labeled data are scarce or the test distribution shifts. Prompt learning adapts only a small set of parameters while keeping the backbone frozen, yet many existing multimodal prompt learners couple the visual and textual branches weakly and can be brittle in low-shot regimes. We propose ProCAP, a probabilistic cross-attentive prompt learning framework that improves cross-modal interaction and training stability without updating any CLIP weights: it learns both visual and textual prompt tokens and links them through stacked bidirectional multi-head cross-attention so the two branches refine each other across prompt depth. To reduce overfitting under limited supervision, we parameterize prompt tokens with Gaussian means and variances and regularize them with lightweight KL and L2 penalties, and we further add a compact symmetric InfoNCE head that aligns cross-attended image features with class-level text representations in a shared low-dimensional space. Across few-shot base-to-novel generalization on 11 datasets, cross-dataset transfer, and domain generalization on ImageNet shift benchmarks, ProCAP achieves strong aggregate base-to-novel performance and competitive transfer performance while keeping the CLIP backbone unchanged.