mlsanityData to 14 Oct 2026

Structured Prompt Tuning

arXiv 2205.12309 ↗First posted 24 May 2022cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

We propose structured prompt tuning, a simple and effective method to improve prompt tuning. Instead of prepending a sequence of tunable embeddings to the input, we generate the soft prompt embeddings through a hypernetwork. Our approach subsumes the standard prompt tuning, allows more flexibility in model design and can be applied to both single-task and multi-task training settings. Empirically, structured prompt tuning shows a gain of +1.2~1.5 points on the GLUE benchmark and is less sensitive to the change of learning rate, compared to standard prompt tuning.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 18 of its 20 references that are in the map. First posted on arXiv 24 May 2022.

Builds on

18 of 20 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  11. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  12. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
6 more
  1. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  2. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 669
  3. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 377
  4. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  5. 2021WARP: Word-level Adversarial ReProgrammingcited by 221
  6. 2022Standing on the Shoulders of Giant Frozen Language Modelscited by 33

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 1.

4 papers.

  1. 2023SpeechPrompt v2: Prompt Tuning for Speech Classification Taskscited by 32
  2. 2023InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understandingcited by 30
  3. 2025LightPROF: A Lightweight Reasoning Framework for Large Language Model on Knowledge Graphcited by 8
  4. 2024Compositional Kronecker Context Optimization for Vision-Language Modelscited by 0