# A Survey on Prompt Tuning

arXiv: https://arxiv.org/abs/2507.06085 · HTML: https://mlsanity.com/paper/2507.06085/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 32 references that are in the map.
- First posted on arXiv 8 Jul 2025.

## Builds on (most cited first, 24 of 24)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models](https://mlsanity.com/paper/2106.10199/) (2021)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning](https://mlsanity.com/paper/2205.05638/) (2022)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Efficient Transformers: A Survey](https://mlsanity.com/paper/2009.06732/) (2020)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey](https://mlsanity.com/paper/2403.14608/) (2024)
- [SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer](https://mlsanity.com/paper/2110.07904/) (2021)
- [PPT: Pre-trained Prompt Tuning for Few-shot Learning](https://mlsanity.com/paper/2109.04332/) (2021)
- [Efficient Large Language Models: A Survey](https://mlsanity.com/paper/2312.03863/) (2023)
- [CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models](https://mlsanity.com/paper/2109.11797/) (2021)
- [Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2303.02861/) (2023)
- [ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts](https://mlsanity.com/paper/2205.11961/) (2022)
- [Prompt Compression for Large Language Models: A Survey](https://mlsanity.com/paper/2410.12388/) (2024)
- [Residual Prompt Tuning: Improving Prompt Tuning with Residual Reparameterization](https://mlsanity.com/paper/2305.03937/) (2023)
- [500xCompressor: Generalized Prompt Compression for Large Language Models](https://mlsanity.com/paper/2408.03094/) (2024)
- [XPrompt: Exploring the Extreme of Prompt Tuning](https://mlsanity.com/paper/2210.04457/) (2022)
- [PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning](https://mlsanity.com/paper/2505.09519/) (2025)
- [Efficient Cross-Task Prompt Tuning for Few-Shot Conversational Emotion Recognition](https://mlsanity.com/paper/2310.14614/) (2023)

## Built on by (most cited first, 7 of 7)

- [Modular Prompt Optimization: Optimizing Structured Prompts with Section-Local Textual Gradients](https://mlsanity.com/paper/2601.04055/) (2026)
- [Is In-Context Learning Learning?](https://mlsanity.com/paper/2509.10414/) (2025)
- [Breaking Training Bottlenecks: Effective and Stable Reinforcement Learning for Coding Models](https://mlsanity.com/paper/2603.07777/) (2026)
- [LPDS: Evaluating LLM Robustness Through Logic-Preserving Difficulty Scaling](https://mlsanity.com/paper/2605.15393/) (2026)
- [AutoScientist-Quant: Self-Evolving Coding Agents for Automatic Research in Quantitative Investment](https://mlsanity.com/paper/2608.28632/) (2026)
- [AutoFed: Personalized Federated Traffic Prediction via Adaptive Prompt](https://mlsanity.com/paper/2512.24625/) (2025)
- [On the Capability and Limitation of Hard Prompt](https://mlsanity.com/paper/2609.32302/) (2026)

## Abstract (arXiv)

This survey reviews prompt tuning, a parameter-efficient approach for adapting language models by prepending trainable continuous vectors while keeping the model frozen. We classify existing approaches into two categories: direct prompt learning and transfer learning. Direct prompt learning methods include: general optimization approaches, encoder-based methods, decomposition strategies, and mixture-of-experts frameworks. Transfer learning methods consist of: general transfer approaches, encoder-based methods, and decomposition strategies. For each method, we analyze method designs, innovations, insights, advantages, and disadvantages, with illustrative visualizations comparing different frameworks. We identify challenges in computational efficiency and training stability, and discuss future directions in improving training robustness and broadening application scope.