# PPOM: Marginalizing Patch-Grid Phase for CLIP-Based Generalizable Vision-Language Prompt Tuning

arXiv: https://arxiv.org/abs/2608.13969 · HTML: https://mlsanity.com/paper/2608.13969/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 16 of its 17 references that are in the map.
- First posted on arXiv 14 Aug 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 16 of 16)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [MaPLe: Multi-modal Prompt Learning](https://mlsanity.com/paper/2210.03117/) (2022)
- [Visual-Language Prompt Tuning with Knowledge-guided Context Optimization](https://mlsanity.com/paper/2303.13283/) (2023)
- [Self-regulating Prompts: Foundational Model Adaptation without Forgetting](https://mlsanity.com/paper/2307.06948/) (2023)
- [Greedy Policy Search: A Simple Baseline for Learnable Test-Time Augmentation](https://mlsanity.com/paper/2002.09103/) (2020)
- [DePT: Decoupled Prompt Tuning](https://mlsanity.com/paper/2309.07439/) (2023)
- [MMRL: Multi-Modal Representation Learning for Vision-Language Models](https://mlsanity.com/paper/2503.08497/) (2025)
- [Advancing Textual Prompt Learning with Anchored Attributes](https://mlsanity.com/paper/2412.09442/) (2024)
- [Cascade Prompt Learning for Vision-Language Model Adaptation](https://mlsanity.com/paper/2409.17805/) (2024)
- [DPC: Dual-Prompt Collaboration for Tuning Vision-Language Models](https://mlsanity.com/paper/2503.13443/) (2025)
- [Making Vision Transformers Truly Shift-Equivariant](https://mlsanity.com/paper/2305.16316/) (2023)
- [MAO: Efficient Model-Agnostic Optimization of Prompt Tuning for Vision-Language Models](https://mlsanity.com/paper/2503.18160/) (2025)
- [Raw Data Matters: Enhancing Prompt Tuning by Internal Augmentation on Vision-Language Models](https://mlsanity.com/paper/2508.02671/) (2025)
- [FVG-PT: Adaptive Foreground View-Guided Prompt Tuning for Vision-Language Models](https://mlsanity.com/paper/2603.08708/) (2026)

## Abstract (arXiv)

Prompt tuning adapts CLIP-based vision-language models with few trainable parameters, yet its predictions remain sensitive to the spatial sampling imposed by a frozen vision transformer. In particular, non-overlapping patch tokenization makes predictions depend on the alignment (phase) between image and the patch lattice. To reduce prediction sensitivity to patch-grid alignment, we introduce Patch-Phase Orbit Marginalization (PPOM), a training-free inference operator that treats phase shift as a nuisance variable. Given a patch stride, PPOM evaluates the identity view and reflection-padded translations, pairs opposite shifts into horizontal, vertical, and diagonal antithetic families, and assigns equal mass to these families and the identity prediction to avoid view-count bias during phase integration. In summary, PPOM provides a deterministic interface between prompt adaptation and patch-grid sensitivity. Across multiple prompt-learning hosts, PPOM improves host performance without re-training.