mlsanityData to 7 Oct 2026

PPOM: Marginalizing Patch-Grid Phase for CLIP-Based Generalizable Vision-Language Prompt Tuning

arXiv 2608.13969 ↗First posted 14 Aug 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Prompt tuning adapts CLIP-based vision-language models with few trainable parameters, yet its predictions remain sensitive to the spatial sampling imposed by a frozen vision transformer. In particular, non-overlapping patch tokenization makes predictions depend on the alignment (phase) between image and the patch lattice. To reduce prediction sensitivity to patch-grid alignment, we introduce Patch-Phase Orbit Marginalization (PPOM), a training-free inference operator that treats phase shift as a nuisance variable. Given a patch stride, PPOM evaluates the identity view and reflection-padded translations, pairs opposite shifts into horizontal, vertical, and diagonal antithetic families, and assigns equal mass to these families and the identity prediction to avoid view-count bias during phase integration. In summary, PPOM provides a deterministic interface between prompt adaptation and patch-grid sensitivity. Across multiple prompt-learning hosts, PPOM improves host performance without re-training.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 16 of its 17 references that are in the map. First posted on arXiv 14 Aug 2026.

Builds on

16 of 17 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026FVG-PT: Adaptive Foreground View-Guided Prompt Tuning for Vision-Language Models9 Mar
  2. 2025Raw Data Matters: Enhancing Prompt Tuning by Internal Augmentation on Vision-Language Models4 Aug
  3. 2025MAO: Efficient Model-Agnostic Optimization of Prompt Tuning for Vision-Language Models23 Mar
  4. 2025DPC: Dual-Prompt Collaboration for Tuning Vision-Language Models17 Mar
  5. 2024Cascade Prompt Learning for Vision-Language Model Adaptation26 Sep
  6. 2023Visual-Language Prompt Tuning with Knowledge-guided Context Optimization23 Mar
  7. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 9

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  4. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  5. 2020Greedy Policy Search: A Simple Baseline for Learnable Test-Time Augmentationcited by 61
  6. 2023DePT: Decoupled Prompt Tuningcited by 58
  7. 2025MMRL: Multi-Modal Representation Learning for Vision-Language Modelscited by 38
  8. 2024Advancing Textual Prompt Learning with Anchored Attributescited by 20
  9. 2023Making Vision Transformers Truly Shift-Equivariantcited by 16

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ZOMP: Zeroth-Order Multi-Modal Prompt Tuning for Vision-Language Models
  2. 2023Consistency-guided Prompt Learning for Vision-Language Models
  3. 2023Patch-Prompt Aligned Bayesian Prompt Tuning for Vision-Language Models
  4. 2023Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models
  5. 2025ZIP: An Efficient Zeroth-order Prompt Tuning for Black-box Vision-Language Models
  6. 2026Towards Calibrating Prompt Tuning of Vision-Language Models
  7. 2024PromptSync: Bridging Domain Gaps in Vision-Language Models through Class-Aware Prototype Alignment and Discrimination
  8. 2026Intrinsic Gradient Suppression for Label-Noise Prompt Tuning in Vision-Language Models