mlsanityData to 7 Oct 2026

Robustness of Demonstration-based Learning Under Limited Data Scenario

arXiv 2210.10693 ↗First posted 19 Oct 2022cs.CL cs.AI cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Demonstration-based learning has shown great potential in stimulating pretrained language models' ability under limited data scenario. Simply augmenting the input with some demonstrations can significantly improve performance on few-shot NER. However, why such demonstrations are beneficial for the learning process remains unclear since there is no explicit alignment between the demonstrations and the predictions. In this paper, we design pathological demonstrations by gradually removing intuitively useful information from the standard ones to take a deep dive of the robustness of demonstration-based sequence labeling and show that (1) demonstrations composed of random tokens still make the model a better few-shot learner; (2) the length of random demonstrations and the relevance of random tokens are the main factors affecting the performance; (3) demonstrations increase the confidence of model predictions on captured superficial patterns. We have publicly released our code at https://github.com/SALT-NLP/RobustDemo.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 30 of its 42 references that are in the map. First posted on arXiv 19 Oct 2022.

Builds on

30 of 42 references.

References · 30

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  9. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  10. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  11. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  12. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
18 more
  1. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  2. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  3. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  4. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
  5. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  6. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  7. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  8. 2021Noisy Channel Language Model Prompting for Few-Shot Text Classificationcited by 163
  9. 2021Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Modelscited by 139
  10. 2022Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrationscited by 116
  11. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  12. 2021Template-free Prompt Tuning for Few-shot NERcited by 73
  13. 2021Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NERcited by 38
  14. 2022Pretraining with Artificial Language: Studying Transferable Knowledge in Language Modelscited by 28
  15. 2021On the Transferability of Pre-trained Language Models: A Study from Artificial Datasetscited by 25
  16. 2021Avoiding Inference Heuristics in Few-shot Prompt-based Finetuningcited by 23
  17. 2021Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognitioncited by 19
  18. 2022Contrastive Demonstration Tuning for Pre-trained Language Modelscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 1; 2025: 2; 2026: 1.

11 papers.

  1. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
  2. 2023Multilingual Machine Translation with Large Language Models: Empirical Results and Analysiscited by 156
  3. 2023Understanding In-Context Learning via Supportive Pretraining Datacited by 41
  4. 2023The Life Cycle of Knowledge in Big Language Models: A Surveycited by 23
  5. 2025Thinking beyond the anthropomorphic paradigm benefits LLM researchcited by 13
  6. 2024Implicit In-context Learningcited by 11
  7. 2023Learning In-context Learning for Named Entity Recognitioncited by 7
  8. 2026NovBench: Evaluating Large Language Models on Academic Paper Novelty Assessmentcited by 6
  9. 2023DemoSG: Demonstration-enhanced Schema-guided Generation for Low-resource Event Extractioncited by 5
  10. 2025Can LLMs Help Uncover Insights about LLMs? A Large-Scale, Evolving Literature Analysis of Frontier LLMscited by 3
  11. 2023Improving a Named Entity Recognizer Trained on Noisy Data with a Few Clean Instancescited by 0