# Raw Data Matters: Enhancing Prompt Tuning by Internal Augmentation on Vision-Language Models

arXiv: https://arxiv.org/abs/2508.02671 · HTML: https://mlsanity.com/paper/2508.02671/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 10 of its 50 references that are in the map.
- First posted on arXiv 4 Aug 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 10 of 10)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey](https://mlsanity.com/paper/2403.14608/) (2024)
- [Unified Vision and Language Prompt Learning](https://mlsanity.com/paper/2210.07225/) (2022)
- [Unsupervised Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2204.03649/) (2022)
- [MMRL: Multi-Modal Representation Learning for Vision-Language Models](https://mlsanity.com/paper/2503.08497/) (2025)
- [Advancing Textual Prompt Learning with Anchored Attributes](https://mlsanity.com/paper/2412.09442/) (2024)
- [DPC: Dual-Prompt Collaboration for Tuning Vision-Language Models](https://mlsanity.com/paper/2503.13443/) (2025)
- [MAO: Efficient Model-Agnostic Optimization of Prompt Tuning for Vision-Language Models](https://mlsanity.com/paper/2503.18160/) (2025)

## Built on by (most cited first, 3 of 3)

- [FVG-PT: Adaptive Foreground View-Guided Prompt Tuning for Vision-Language Models](https://mlsanity.com/paper/2603.08708/) (2026)
- [Decoupling Augmentation Bias in Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2511.03367/) (2025)
- [PPOM: Marginalizing Patch-Grid Phase for CLIP-Based Generalizable Vision-Language Prompt Tuning](https://mlsanity.com/paper/2608.13969/) (2026)

## Abstract (arXiv)

For CLIP-based prompt tuning, introducing more data as additional knowledge for enhancing fine-tuning process is proved to be an effective approach. Existing data amplification strategies for prompt tuning typically rely on external knowledge (e.g., large language models or pre-structured knowledge bases), resulting in higher costs for data collection and processing, while generally ignoring further utilization of features in image modality. To address this, we propose Augmentation-driven Prompt Tuning (AugPT), a self-contained distillation-based prompt tuning approach using only internal augmentation on raw dataset to better exploit known features. Specifically, AugPT employs self-supervised augmentation on unlabeled images in the training set, and introduces a novel gating mechanism based on consensus test, reusing the pre-trained prompt tuning backbone model to spontaneously filter noisy samples, further enhancing the quality of augmented views. Extensive experiments validate that AugPT simultaneously enhances model performance and generalization capability without using appended external knowledge. The code of AugPT is available at: https://github.com/JREion/AugPT .