mlsanityData to 7 Oct 2026

MEAL: Stable and Active Learning for Few-Shot Prompting

arXiv 2211.08358 ↗First posted 15 Nov 2022cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

Few-shot classification has made great strides due to foundation models that, through priming and prompting, are highly effective few-shot learners. However, this approach has high variance both across different sets of few shots (data selection) and across different finetuning runs (run variability). This is problematic not only because it impedes the fair comparison of different approaches, but especially because it makes few-shot learning too unreliable for many real-world applications. To alleviate these issues, we make two contributions for more stable and effective few-shot learning: First, we propose novel ensembling methods and show that they substantially reduce run variability. Second, we introduce a new active learning (AL) criterion for data selection and present the first AL-based approach specifically tailored towards prompt-based learning. In our experiments, we show that our combined method, MEAL (Multiprompt finetuning and prediction Ensembling with Active Learning), improves overall performance of prompt-based finetuning by 2.3 points on five diverse tasks. We publicly share our code and data splits in https://github.com/akoksal/MEAL.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 25 of its 42 references that are in the map. First posted on arXiv 15 Nov 2022.

Builds on

25 of 42 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  4. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  5. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  6. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  7. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  8. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  9. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  10. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  11. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  12. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
13 more
  1. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  2. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  3. 2020On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselinescited by 254
  4. 2021Active Learning by Acquiring Contrastive Examplescited by 154
  5. 2013Stabilitycited by 119
  6. 2021Improving and Simplifying Pattern Exploiting Trainingcited by 97
  7. 2019Diverse mini-batch Active Learningcited by 88
  8. 2020Few-Shot Text Generation with Pattern-Exploiting Trainingcited by 87
  9. 2021Revisiting Uncertainty-based Query Strategies for Active Learning with Transformerscited by 59
  10. 2020A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matterscited by 45
  11. 2019Sampling Bias in Deep Active Classification: An Empirical Studycited by 36
  12. 2021FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understandingcited by 17
  13. 2022Multi-task Active Learning for Pre-trained Transformer-based Modelscited by 16

Built on by

Cited by year

Citing papers per year: 2023: 7; 2024: 10; 2025: 3.

20 papers.

Sort:

By area:

  1. 2023Active Prompting with Chain-of-Thought for Large Language Modelscited by 154
  2. 2023Active Learning Principles for In-Context Learning with Large Language Modelscited by 53
  3. 2023The language of prompting: What linguistic properties make a prompt successful?cited by 43
  4. 2024Effects of diversity incentives on sample diversity and downstream model performance in LLM-based text augmentationcited by 17
  5. 2024Comparing Specialised Small and General Large Language Models on Text Classification: 100 Labelled Samples to Achieve Break-Even Performancecited by 17
  6. 2023A Survey on Stability of Learning with Limited Labelled Data and its Sensitivity to the Effects of Randomnesscited by 14
  7. 2023MorphPiece : A Linguistic Tokenizer for Large Language Modelscited by 9
  8. 2024LLMs vs Established Text Augmentation Techniques for Classification: When do the Benefits Outweight the Costs?cited by 8
  9. 2024Flexible and Adaptable Summarization via Expertise Separationcited by 7
  10. 2024Use Random Selection for Now: Investigation of Few-Shot Selection Strategies in LLM-based Text Augmentation for Classificationcited by 7
  11. 2023Do prompt positions really matter?cited by 5
  12. 2024Automatic Combination of Sample Selection Strategies for Few-Shot Learningcited by 4
8 more
  1. 2024StablePT: Towards Stable Prompting for Few-shot Learning via Input Separationcited by 3
  2. 2024Fighting Randomness with Randomness: Mitigating Optimisation Instability of Fine-Tuning using Delayed Ensemble and Noisy Interpolationcited by 2
  3. 2025PromptAL: Sample-Aware Dynamic Soft Prompts for Few-Shot Active Learningcited by 2
  4. 2023Sociocultural knowledge is needed for selection of shots in hate speech detection taskscited by 1
  5. 2024On Sensitivity of Learning with Limited Labelled Data to the Effects of Randomness: Impact of Interactions and Systematic Choicescited by 1
  6. 2025Large Language Models are Demonstration Pre-Selectors for Themselvescited by 1
  7. 2024The Power of Adaptation: Boosting In-Context Learning through Adaptive Promptingcited by 0
  8. 2025Active Few-Shot Learning for Vertex Classification Starting from an Unlabeled Datasetcited by 0