mlsanityData to 7 Oct 2026

Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference

arXiv 2001.07676 ↗First posted 21 Jan 2020cs.CLcited by 934

Not on the map. Why?

Cited by 22 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Some NLP tasks can be solved in a fully unsupervised fashion by providing a pretrained language model with "task descriptions" in natural language (e.g., Radford et al., 2019). While this approach underperforms its supervised counterpart, we show in this work that the two ideas can be combined: We introduce Pattern-Exploiting Training (PET), a semi-supervised training procedure that reformulates input examples as cloze-style phrases to help language models understand a given task. These phrases are then used to assign soft labels to a large set of unlabeled examples. Finally, standard supervised training is performed on the resulting training set. For several tasks and languages, PET outperforms supervised training and strong semi-supervised approaches in low-resource settings by a large margin.

Key facts. Cited by 934 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 116; 2022: 271; 2023: 302; 2024: 137; 2025: 67; 2026: 41.

934 papers.

Sort:

By area:

  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  2. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  5. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  6. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  7. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  8. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  9. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  10. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  11. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  12. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
38 more
  1. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  2. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  3. 2022Large Language Models Can Self-Improvecited by 656
  4. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  5. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  6. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  7. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  8. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
  9. 2021True Few-Shot Learning with Language Modelscited by 343
  10. 2023Fine-Tuning Language Models with Just Forward Passescited by 337
  11. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  12. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  13. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  14. 2022Large Language Models Are Reasoning Teacherscited by 318
  15. 2022Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classificationcited by 310
  16. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 309
  17. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 288
  18. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  19. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 273
  20. 2022Exploring Visual Prompts for Adapting Large-Scale Modelscited by 271
  21. 2022What does a platypus look like? Generating customized prompts for zero-shot image classificationcited by 263
  22. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 256
  23. 2022Black-Box Tuning for Language-Model-as-a-Servicecited by 254
  24. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  25. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  26. 2021PPT: Pre-trained Prompt Tuning for Few-shot Learningcited by 232
  27. 2022Zero-Shot Video Question Answering via Frozen Bidirectional Language Modelscited by 229
  28. 2021WARP: Word-level Adversarial ReProgrammingcited by 220
  29. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 213
  30. 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 206
  31. 2023Data-centric Artificial Intelligence: A Surveycited by 202
  32. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  33. 2021How Many Data Points is a Prompt Worth?cited by 197
  34. 2021Generating Datasets with Pretrained Language Modelscited by 192
  35. 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 189
  36. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 186
  37. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 185
  38. 2021Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classificationcited by 183

50 most cited of 934 All 934 (JSON)