mlsanityData to 7 Oct 2026

Entailment as Few-Shot Learner

arXiv 2104.14690 ↗First posted 29 Apr 2021cs.CL cs.AIcited by 122

Not on the map. Why?

Abstract (arXiv)

Large pre-trained language models (LMs) have demonstrated remarkable ability as few-shot learners. However, their success hinges largely on scaling model parameters to a degree that makes it challenging to train and serve. In this paper, we propose a new approach, named as EFL, that can turn small LMs into better few-shot learners. The key idea of this approach is to reformulate potential NLP task into an entailment one, and then fine-tune the model with as little as 8 examples. We further demonstrate our proposed method can be: (i) naturally combined with an unsupervised contrastive learning-based data augmentation method; (ii) easily extended to multilingual few-shot learning. A systematic evaluation on 18 standard NLP tasks demonstrates that this approach improves the various existing SOTA few-shot learning methods by 12%, and yields competitive few-shot performance with 500 times larger models, such as GPT-3.

Key facts. Cited by 122 papers in the map (data through 7 Oct 2026). Builds on 33 of its 51 references that are in the map. First posted on arXiv 29 Apr 2021.

Builds on

33 of 51 references.

References · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  9. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  11. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  12. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
21 more
  1. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  2. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  3. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  4. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  5. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  6. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  7. 2019How Can We Know What Language Models Know?cited by 953
  8. 2018Neural Network Acceptability Judgmentscited by 952
  9. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  10. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  11. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  12. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  13. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 275
  14. 2020Revisiting Few-sample BERT Fine-tuningcited by 237
  15. 2018Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Taskscited by 214
  16. 2018Scaling Neural Machine Translationcited by 204
  17. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  18. 2019Certified Robustness to Adversarial Word Substitutionscited by 164
  19. 2019Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Modelscited by 141
  20. 2019Zero-shot Text Classification With Generative Language Modelscited by 55
  21. 2020Universal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Startcited by 44

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 39; 2023: 32; 2024: 21; 2025: 12; 2026: 2.

122 papers.

Sort:

By area:

  1. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  2. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  3. 2021True Few-Shot Learning with Language Modelscited by 343
  4. 2022GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Recordscited by 238
  5. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  6. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMscited by 211
  7. 2023ChatGPT: Jack of all trades, master of nonecited by 204
  8. 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 162
  9. 2022A Review on Language Models as Knowledge Basescited by 156
  10. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 144
  11. 2021Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learnerscited by 115
  12. 2021Label Verbalization and Entailment for Effective Zero- and Few-Shot Relation Extractioncited by 73
38 more
  1. 2022Black-box Prompt Learning for Pre-trained Language Modelscited by 70
  2. 2022Improving In-Context Few-Shot Learning via Self-Supervised Trainingcited by 56
  3. 2022IDPG: An Instance-Dependent Prompt Generation Methodcited by 43
  4. 2021FewCLUE: A Chinese Few-shot Learning Evaluation Benchmarkcited by 41
  5. 2021Paradigm Shift in Natural Language Processingcited by 39
  6. 2021STraTA: Self-Training with Task Augmentation for Better Few-shot Learningcited by 33
  7. 2023MMG-Ego4D: Multi-Modal Generalization in Egocentric Action Recognitioncited by 26
  8. 2022Textual Entailment for Event Argument Extraction: Zero- and Few-Shot with Multi-Source Learningcited by 25
  9. 2023Mask-guided BERT for Few Shot Text Classificationcited by 25
  10. 2021Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanationscited by 24
  11. 2021NSP-BERT: A Prompt-based Few-Shot Learner Through an Original Pre-training Task--Next Sentence Predictioncited by 23
  12. 2021Open Aspect Target Sentiment Classification with Natural Language Promptscited by 20
  13. 2021Interpreting Deep Learning Models in Natural Language Processing: A Reviewcited by 19
  14. 2023Building Efficient Universal Classifiers with Natural Language Inferencecited by 19
  15. 2021Monolingual and Cross-Lingual Acceptability Judgments with the Italian CoLA corpuscited by 18
  16. 2022Towards Unified Prompt Tuning for Few-shot Text Classificationcited by 18
  17. 2023Moderating New Waves of Online Hate with Chain-of-Thought Reasoning in Large Language Modelscited by 17
  18. 2024GeAR: Graph-enhanced Agent for Retrieval-augmented Generationcited by 17
  19. 2022In-BoXBART: Get Instructions into Biomedical Multi-Task Learningcited by 16
  20. 2022Y-Tuning: An Efficient Tuning Paradigm for Large-Scale Pre-Trained Models via Label Representation Learningcited by 15
  21. 2022Few-Shot Learning with Siamese Networks and Label Tuningcited by 15
  22. 2023FACTIFY3M: A Benchmark for Multimodal Fact Verification with Explainability through 5W Question-Answeringcited by 15
  23. 2022PART: Pre-trained Authorship Representation Transformercited by 13
  24. 2024Lessons from the Use of Natural Language Inference (NLI) in Requirements Engineering Taskscited by 12
  25. 2025Zero-shot and Few-shot Learning with Instruction-following LLMs for Claim Matching in Automated Fact-checkingcited by 12
  26. 2022Active Few-Shot Learning with FASLcited by 11
  27. 2023Sentiment Analysis through LLM Negotiationscited by 11
  28. 2024ClaimVer: Explainable Claim-Level Verification and Evidence Attribution of Text Through Knowledge Graphscited by 11
  29. 2025Taxonomy-Aware Evaluation of Vision-Language Modelscited by 11
  30. 2023Scalable Prompt Generation for Semi-supervised Learning with Language Modelscited by 10
  31. 2023SEED: Domain-Specific Data Curation With Large Language Modelscited by 10
  32. 2023What do Language Models know about word senses? Zero-Shot WSD with Language Models and Domain Inventoriescited by 9
  33. 2021PECO: Examining Single Sentence Label Leakage in Natural Language Inference Datasets through Progressive Evaluation of Cluster Outlierscited by 8
  34. 2023Adversarial Robustness of Prompt-based Few-Shot Learning for Natural Language Understandingcited by 8
  35. 2023Text Alignment Is An Efficient Unified Model for Massive NLP Taskscited by 8
  36. 2021Explainable Natural Language Processing with Matrix Product Statescited by 7
  37. 2022Pre-trained Token-replaced Detection Model as Few-shot Learnercited by 7
  38. 2022ZS4IE: A toolkit for Zero-Shot Information Extraction with simple Verbalizationscited by 7

50 most cited of 122 All 122 (JSON)