mlsanityData to 14 Oct 2026

Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations

arXiv 2212.09865 ↗First posted 19 Dec 2022cs.CL cs.AIcited by 70

Not on the map. Why?

Abstract (arXiv)

Although large language models can be prompted for both zero- and few-shot learning, performance drops significantly when no demonstrations are available. In this paper, we introduce Z-ICL, a new zero-shot method that closes the gap by constructing pseudo-demonstrations for a given test input using a raw text corpus. Concretely, pseudo-demonstrations are constructed by (1) finding the nearest neighbors to the test input from the corpus and pairing them with random task labels, and (2) applying a set of techniques to reduce the amount of direct copying the model does from the resulting demonstrations. Evaluation on nine classification datasets shows that Z-ICL outperforms previous zero-shot methods by a significant margin, and is on par with in-context learning with labeled training data in the few-shot setting. Overall, Z-ICL provides a significantly higher estimate of the zero-shot performance levels of a model, and supports future efforts to develop better pseudo-demonstrations that further improve zero-shot results.

Key facts. Cited by 70 papers in the map (data through 14 Oct 2026). Builds on 20 of its 27 references that are in the map. First posted on arXiv 19 Dec 2022.

Builds on

20 of 27 references.

References · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2017Billion-scale similarity search with GPUscited by 2,497
  4. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  5. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  6. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  7. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  8. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  9. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  10. 2022In-context Learning and Induction Headscited by 964
  11. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  12. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
8 more
  1. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  2. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  3. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  4. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 417
  5. 2021Noisy Channel Language Model Prompting for Few-Shot Text Classificationcited by 165
  6. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
  7. 2021DEMix Layers: Disentangling Domains for Modular Language Modelingcited by 132
  8. 2022Semantic-Oriented Unlabeled Priming for Large-Scale Language Modelscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 24; 2024: 26; 2025: 16; 2026: 4.

70 papers.

Sort:

By area:

  1. 2023The Learnability of In-Context Learningcited by 127
  2. 2024Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvementscited by 83
  3. 2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
  4. 2024A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasonerscited by 73
  5. 2023Dr.ICL: Demonstration-Retrieved In-context Learningcited by 72
  6. 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 70
  7. 2023Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Modelscited by 61
  8. 2024Dual Operating Modes of In-Context Learningcited by 43
  9. 2023Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrationscited by 42
  10. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  11. 2023MoT: Memory-of-Thought Enables ChatGPT to Self-Improvecited by 38
  12. 2023Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Followingcited by 33
38 more
  1. 2023How to Configure Good In-Context Sequence for Visual Question Answeringcited by 33
  2. 2024LLMs Are In-Context Bandit Reinforcement Learnerscited by 29
  3. 2023Large Language Models are Strong Zero-Shot Retrievercited by 26
  4. 2024C-ICL: Contrastive In-context Learning for Information Extractioncited by 23
  5. 2025TACO: Enhancing Multimodal In-context Learning via Task Mapping-Guided Sequence Configurationcited by 23
  6. 2023IDAS: Intent Discovery with Abstractive Summarizationcited by 22
  7. 2023Supervised Knowledge Makes Large Language Models Better In-context Learnerscited by 20
  8. 2023Dissecting Chain-of-Thought: Compositionality through In-Context Filtering and Learningcited by 19
  9. 2023Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruningcited by 17
  10. 2024On the Noise Robustness of In-Context Learning for Text Generationcited by 16
  11. 2023Confronting LLMs with Traditional ML: Rethinking the Fairness of Large Language Models in Tabular Classificationscited by 15
  12. 2023Self-Improving for Zero-Shot Named Entity Recognition with Large Language Modelscited by 15
  13. 2024Exploring Automated Distractor Generation for Math Multiple-choice Questions via Large Language Modelscited by 15
  14. 2023Universal Self-Adaptive Promptingcited by 14
  15. 2024Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superpositioncited by 14
  16. 2023Improving Probability-based Prompt Selection Through Unified Evaluation and Analysiscited by 13
  17. 2024The Impact of Demonstrations on Multilingual In-Context Learning: A Multidimensional Analysiscited by 13
  18. 2024Language Models can Exploit Cross-Task In-context Learning for Data-Scarce Novel Taskscited by 13
  19. 2024In-Context Example Ordering Guided by Label Distributionscited by 12
  20. 2024Retrieval-Augmented Data Augmentation for Low-Resource Domain Taskscited by 12
  21. 2023Ambiguity-Aware In-Context Learning with Large Language Modelscited by 11
  22. 2024Tox-BART: Leveraging Toxicity Attributes for Explanation Generation of Implicit Hate Speechcited by 11
  23. 2024DRUM: Learning Demonstration Retriever for Large MUlti-modal Modelscited by 10
  24. 2025EvaLearn: Quantifying the Learning Capability and Efficiency of LLMs via Sequential Problem Solvingcited by 10
  25. 2023Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learningcited by 9
  26. 2023Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptionscited by 9
  27. 2024A Survey of Data Synthesis Approachescited by 9
  28. 2024Self-Evolving GPT: A Lifelong Autonomous Experiential Learnercited by 9
  29. 2023Scaling In-Context Demonstrations with Structured Attentioncited by 8
  30. 2024Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learningcited by 8
  31. 2024Relevant or Random: Can LLMs Truly Perform Analogical Reasoning?cited by 8
  32. 2025Advancing Multimodal In-Context Learning in Large Vision-Language Models with Task-aware Demonstrationscited by 7
  33. 2023Knowledgeable In-Context Tuning: Exploring and Exploiting Factual Knowledge for In-Context Learningcited by 6
  34. 2024DAWN-ICL: Strategic Planning of Problem-solving Trajectories for Zero-Shot In-Context Learningcited by 6
  35. 2023ICXML: An In-Context Learning Framework for Zero-Shot Extreme Multi-Label Classificationcited by 5
  36. 2024P-ICL: Point In-Context Learning for Named Entity Recognition with Large Language Modelscited by 5
  37. 2024MM-PoE: Multiple Choice Reasoning via. Process of Elimination using Multi-Modal Modelscited by 5
  38. 2025Collaborative Evolution: Multi-Round Learning Between Large and Small Language Models for Emergent Fake News Detectioncited by 5

50 most cited of 70 All 70 (JSON)