mlsanityData to 7 Oct 2026

Prototypical Verbalizer for Prompt-based Few-shot Tuning

arXiv 2203.09770 ↗First posted 18 Mar 2022cs.CL cs.LGcited by 53

Not on the map. Why?

Abstract (arXiv)

Prompt-based tuning for pre-trained language models (PLMs) has shown its effectiveness in few-shot learning. Typically, prompt-based tuning wraps the input text into a cloze question. To make predictions, the model maps the output words to labels via a verbalizer, which is either manually designed or automatically built. However, manual verbalizers heavily depend on domain-specific prior knowledge and human efforts, while finding appropriate label words automatically still remains challenging.In this work, we propose the prototypical verbalizer (ProtoVerb) which is built directly from training data. Specifically, ProtoVerb learns prototype vectors as verbalizers by contrastive learning. In this way, the prototypes summarize training instances and are able to enclose rich class-level semantics. We conduct experiments on both topic classification and entity typing tasks, and the results demonstrate that ProtoVerb significantly outperforms current automatic verbalizers, especially when training data is extremely scarce. More surprisingly, ProtoVerb consistently boosts prompt-based tuning even on untuned PLMs, indicating an elegant non-tuning way to utilize PLMs. Our codes are avaliable at https://github.com/thunlp/OpenPrompt.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 32 of its 44 references that are in the map. First posted on arXiv 18 Mar 2022.

Builds on

32 of 44 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  11. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  12. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
20 more
  1. 2019Language Models as Knowledge Bases?cited by 1,922
  2. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  3. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  4. 2021GPT Understands, Toocited by 715
  5. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 636
  6. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  7. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  8. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  9. 2018A Simple Method for Commonsense Reasoningcited by 223
  10. 2021WARP: Word-level Adversarial ReProgrammingcited by 220
  11. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  12. 2021Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classificationcited by 183
  13. 2019Text Classification Algorithms: A Surveycited by 163
  14. 2021Few-NERD: A Few-Shot Named Entity Recognition Datasetcited by 150
  15. 2021OpenPrompt: An Open-source Framework for Prompt-learningcited by 148
  16. 2021Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learnerscited by 115
  17. 2020Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classificationcited by 98
  18. 2021FLEX: Unifying Evaluation for Few-Shot NLPcited by 67
  19. 2021Prompt-Learning for Fine-Grained Entity Typingcited by 62
  20. 2021Prototypical Representation Learning for Relation Extractioncited by 33

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 24; 2024: 13; 2025: 4; 2026: 2.

53 papers.

Sort:

By area:

  1. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 273
  2. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
  3. 2023Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Trainingcited by 85
  4. 2022A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarkscited by 70
  5. 2022Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learningcited by 36
  6. 2023SGL-PT: A Strong Graph Learner with Graph Prompt Tuningcited by 35
  7. 2024Generative AI for Synthetic Data Generation: Methods, Challenges and the Futurecited by 31
  8. 2024Beyond Performance: Quantifying and Mitigating Label Bias in LLMscited by 24
  9. 2023Language models are weak learnerscited by 21
  10. 2022MetaPrompting: Learning to Learn Better Promptscited by 19
  11. 2022Few-shot Named Entity Recognition with Entity-level Prototypical Network Enhanced by Dispersedly Distributed Prototypescited by 17
  12. 2023Effective Structured Prompting by Meta-Learning and Representative Verbalizercited by 14
38 more
  1. 2024SpeechPrompt: Prompting Speech Language Models for Speech Processing Taskscited by 12
  2. 2022Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representationscited by 11
  3. 2023Hierarchical Verbalizer for Few-Shot Hierarchical Text Classificationcited by 11
  4. 2023Scalable Prompt Generation for Semi-supervised Learning with Language Modelscited by 10
  5. 2024TELLER: A Trustworthy Framework for Explainable, Generalizable and Controllable Fake News Detectioncited by 10
  6. 2023Automated Few-shot Classification with Instruction-Finetuned Language Modelscited by 9
  7. 2023Open-vocabulary Video Question Answering: A New Benchmark for Evaluating the Generalizability of Video Question Answering Modelscited by 7
  8. 2022Decoder Tuning: Efficient Language Understanding as Decodingcited by 6
  9. 2023Prototype-based HyperAdapter for Sample-Efficient Multi-task Tuningcited by 6
  10. 2023Clustering-Aware Negative Sampling for Unsupervised Sentence Representationcited by 5
  11. 2023Decoupling Pseudo Label Disambiguation and Representation Learning for Generalized Intent Discoverycited by 5
  12. 2023Gotta: Generative Few-shot Question Answering by Prompt-based Cloze Data Augmentationcited by 5
  13. 2023Rethinking Visual Prompt Learning as Masked Visual Token Modelingcited by 4
  14. 2023Transductive Learning for Textual Few-Shot Classification in API-based Embedding Modelscited by 4
  15. 2023Evolutionary Verbalizer Search for Prompt-based Few Shot Text Classificationcited by 3
  16. 2023APP: Adaptive Prototypical Pseudo-Labeling for Few-shot OOD Detectioncited by 3
  17. 2024Prompt-tuning for Clickbait Detection via Text Summarizationcited by 3
  18. 2024StablePT: Towards Stable Prompting for Few-shot Learning via Input Separationcited by 3
  19. 2022Few-shot Classification with Hypersphere Modeling of Prototypescited by 2
  20. 2022CCPrefix: Counterfactual Contrastive Prefix-Tuning for Many-Class Classificationcited by 2
  21. 2025Hierarchy-Consistent Learning and Adaptive Loss Balancing for Hierarchical Multi-Label Classificationcited by 2
  22. 2023Meta-Learning Triplet Network with Adaptive Margins for Few-Shot Named Entity Recognitioncited by 1
  23. 2023PVP: Pre-trained Visual Parameter-Efficient Tuningcited by 1
  24. 2023Low-Resource Multi-Granularity Academic Function Recognition Based on Multiple Prompt Knowledgecited by 1
  25. 2023MsPrompt: Multi-step Prompt Learning for Debiasing Few-shot Event Detectioncited by 1
  26. 2023Manifold-based Verbalizer Space Re-embedding for Tuning-free Prompt-based Classificationcited by 1
  27. 2024X-Shot: A Unified System to Handle Frequent, Few-shot and Zero-shot Learning Simultaneously in Classificationcited by 1
  28. 2024FPT: Feature Prompt Tuning for Few-shot Readability Assessmentcited by 1
  29. 2025Semantic Anchors in In-Context Learning: Why Small LLMs Cannot Flip Their Labelscited by 1
  30. 2022Technical Report -- Competition Solution for Prompt Tuning using Pretrained Language Modelcited by 0
  31. 2023MetricPrompt: Prompting Model as a Relevance Metric for Few-shot Text Classificationcited by 0
  32. 2023Prompt2Gaussia: Uncertain Prompt-learning for Script Event Predictioncited by 0
  33. 2023Boosting Prompt-Based Self-Training With Mapping-Free Automatic Verbalizer for Multi-Class Classificationcited by 0
  34. 2024A Novel Prompt-tuning Method: Incorporating Scenario-specific Concepts into a Verbalizercited by 0
  35. 2024Manual Verbalizer Enrichment for Few-Shot Text Classificationcited by 0
  36. 2024Improve Meta-learning for Few-Shot Text Classification with All You Can Acquire from the Taskscited by 0
  37. 2024Label Set Optimization via Activation Distribution Kurtosis for Zero-shot Classification with Generative Modelscited by 0
  38. 2025Modeling Data Diversity for Joint Instance and Verbalizer Selection in Cold-Start Scenarioscited by 0

50 most cited of 53 All 53 (JSON)