mlsanityData to 13 Oct 2026

Prototypical Calibration for Few-shot Learning of Language Models

arXiv 2205.10183 ↗First posted 20 May 2022cs.CLcited by 58

Not on the map. Why?

Abstract (arXiv)

In-context learning of GPT-like models has been recognized as fragile across different hand-crafted templates, and demonstration permutations. In this work, we propose prototypical calibration to adaptively learn a more robust decision boundary for zero- and few-shot classification, instead of greedy decoding. Concretely, our method first adopts Gaussian mixture distribution to estimate the prototypical clusters for all categories. Then we assign each cluster to the corresponding label by solving a weighted bipartite matching problem. Given an example, its prediction is calibrated by the likelihood of prototypical clusters. Experimental results show that prototypical calibration yields a substantial improvement on a diverse set of tasks. Extensive analysis across different scales also indicates that our method calibrates the decision boundary as expected, greatly improving the robustness of GPT to templates, permutations, and class imbalance.

Key facts. Cited by 58 papers in the map (data through 13 Oct 2026). Builds on 20 of its 32 references that are in the map. First posted on arXiv 20 May 2022.

Builds on

20 of 32 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2020Language Models are Few-Shot Learnerscited by 32,320
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,884
  4. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,810
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,319
  6. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,424
  7. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,376
  8. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  9. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,191
  10. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,126
  11. 2019How Can We Know What Language Models Know?cited by 958
  12. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 925
8 more
  1. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 670
  2. 2021Learning To Retrieve Prompts for In-Context Learningcited by 643
  3. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 210
  4. 2020UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Trainingcited by 204
  5. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  6. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
  7. 2022On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Modelcited by 81
  8. 2021Towards Zero-Label Language Learningcited by 76

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 5; 2023: 13; 2024: 20; 2025: 15; 2026: 5.

58 papers.

Sort:

By area:

  1. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 457
  2. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 255
  3. 2024In-Context Learning with Long-Context Models: An In-Depth Explorationcited by 128
  4. 2023Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineeringcited by 92
  5. 2024Long-Context Language Modeling with Parallel Context Encodingcited by 84
  6. 2024Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvementscited by 83
  7. 2022Structured Prompting: Scaling In-Context Learning to 1,000 Examplescited by 81
  8. 2022Parallel Context Windows for Large Language Modelscited by 81
  9. 2023Calibration in Deep Learning: A Survey of the State-of-the-Artcited by 73
  10. 2022On the Relation between Sensitivity and Accuracy in In-context Learningcited by 72
  11. 2022Complementary Explanations for Effective In-Context Learningcited by 71
  12. 2023Pre-Training to Learn in Contextcited by 42
38 more
  1. 2024Thermometer: Towards Universal Calibration for Large Language Modelscited by 38
  2. 2023The Consensus Game: Language Model Generation via Equilibrium Searchcited by 36
  3. 2024UniBias: Unveiling and Mitigating LLM Bias through Internal Attention and FFN Manipulationcited by 30
  4. 2025Restoring Calibration for Aligned Large Language Models: A Calibration-Aware Fine-Tuning Approachcited by 25
  5. 2024Beyond Performance: Quantifying and Mitigating Label Bias in LLMscited by 24
  6. 2023A Study on the Calibration of In-context Learningcited by 21
  7. 2023Generative Calibration for In-context Learningcited by 20
  8. 2024Enhancing In-context Learning via Linear Probe Calibrationcited by 16
  9. 2023Universal Self-Adaptive Promptingcited by 14
  10. 2023Unsupervised Contrast-Consistent Ranking with Language Modelscited by 13
  11. 2025Efficient Many-Shot In-Context Learning with Dynamic Block-Sparse Attentioncited by 12
  12. 2024Mitigating Copy Bias in In-Context Learning through Neuron Pruningcited by 11
  13. 2024Confidence Calibration of Classifiers with Many Classescited by 11
  14. 2024Token-based Decision Criteria Are Suboptimal in In-context Learningcited by 9
  15. 2025Incentivizing Truthful Language Models via Peer Elicitation Gamescited by 9
  16. 2025Measuring Scalar Constructs in Social Science with LLMscited by 9
  17. 2023Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptionscited by 8
  18. 2024Shortcut Learning in In-Context Learning: A Surveycited by 8
  19. 2024Prompt-Based Bias Calibration for Better Zero/Few-Shot Learning of Language Modelscited by 7
  20. 2022Decoder Tuning: Efficient Language Understanding as Decodingcited by 6
  21. 2023A Benchmark on Extremely Weakly Supervised Text Classification: Reconcile Seed Matching and Prompting Approachescited by 6
  22. 2024Enhancing In-Context Learning via Implicit Demonstration Augmentationcited by 6
  23. 2024Memorization in In-Context Learningcited by 6
  24. 2024Naive Bayes-based Context Extension for Large Language Modelscited by 5
  25. 2025Exposing and Mitigating Calibration Biases and Demographic Unfairness in MLLM Few-Shot In-Context Learning for Medical Image Classificationcited by 5
  26. 2024Promptly Predicting Structures: The Return of Inferencecited by 4
  27. 2025Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Reviewcited by 3
  28. 2023Matching Table Metadata with Business Glossaries Using Large Language Modelscited by 2
  29. 2023Improving Input-label Mapping with Demonstration Replay for In-context Learningcited by 2
  30. 2024Text Grafting: Near-Distribution Weak Supervision for Minority Classes in Text Classificationcited by 2
  31. 2025StaICC: Standardized Evaluation for Classification Task in In-context Learningcited by 2
  32. 2025Considering Length Diversity in Retrieval-Augmented Summarizationcited by 2
  33. 2025MateICL: Mitigating Attention Dispersion in Large-Scale In-Context Learningcited by 2
  34. 2025Aligning Black-box Language Models with Human Judgmentscited by 1
  35. 2025Boosting In-Context Learning in LLMs Through the Lens of Classical Supervised Learningcited by 1
  36. 2025Surprise Calibration for Better In-Context Learningcited by 1
  37. 2025Mitigating Label Length Bias in Large Language Modelscited by 1
  38. 2026Speaking in Self-Assessing Tongues: On the Verbalized Confidence of LLMs in Machine Translationcited by 1

50 most cited of 58 All 58 (JSON)