Global Prompt Cell: A Portable Control Module for Effective Prompt Tuning
Not on the map. Why?
Abstract (arXiv)
As a novel approach to tuning pre-trained models, prompt tuning involves freezing the parameters in downstream tasks while inserting trainable embeddings into inputs in the first layer. However, previous methods have mainly focused on the initialization of prompt embeddings. The strategy of training and utilizing prompt embeddings in a reasonable way has become a limiting factor in the effectiveness of prompt tuning. To address this issue, we introduce the Global Prompt Cell (GPC), a portable control module for prompt tuning that selectively preserves prompt information across all encoder layers. Our experimental results demonstrate a 5.8% improvement on SuperGLUE datasets compared to vanilla prompt tuning.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 28 references that are in the map. First posted on arXiv 12 Apr 2023.
Builds on
24 of 28 references.
References · 24
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
- 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
- 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
- 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
- 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
- 2014Deep Learning in Neural Networks: An Overviewcited by 770
12 more
- 2021GPT Understands, Toocited by 716
- 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 689
- 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 669
- 2020Pre-trained Models for Natural Language Processing: A Surveycited by 505
- 2018Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Networkcited by 420
- 2019How to Fine-Tune BERT for Text Classification?cited by 411
- 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 377
- 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
- 2021PPT: Pre-trained Prompt Tuning for Few-shot Learningcited by 232
- 2021WARP: Word-level Adversarial ReProgrammingcited by 221
- 2020Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNetscited by 167
- 2021On Transferability of Prompt Tuning for Natural Language Processingcited by 75
Built on by
Citing papers per year: 2025: 1.
1 paper.