mlsanityData to 7 Oct 2026

Memorization in In-Context Learning

arXiv 2408.11546 ↗First posted 21 Aug 2024cs.CL cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

In-context learning (ICL) has proven to be an effective strategy for improving the performance of large language models (LLMs) with no additional training. However, the exact mechanism behind this performance improvement remains unclear. This study is the first to show how ICL surfaces memorized training data and to explore the correlation between this memorization and performance on downstream tasks across various ICL regimes: zero-shot, few-shot, and many-shot. Our most notable findings include: (1) ICL significantly surfaces memorization compared to zero-shot learning in most cases; (2) demonstrations, without their labels, are the most effective element in surfacing memorization; (3) ICL improves performance when the surfaced memorization in few-shot regimes reaches a high level (about 40%); and (4) there is a very strong correlation between performance and memorization in ICL when it outperforms zero-shot learning. Overall, our study uncovers memorization as a new factor impacting ICL, raising an important question: to what extent do LLMs truly generalize from demonstrations in ICL, and how much of their success is due to memorization?

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 47 of its 67 references that are in the map. First posted on arXiv 21 Aug 2024.

Builds on

47 of 67 references.

References · 47

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  5. 2020Extracting Training Data from Large Language Modelscited by 2,144
  6. 2019Language Models as Knowledge Bases?cited by 1,922
  7. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  9. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  10. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  11. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  12. 2022Quantifying Memorization Across Neural Language Modelscited by 755
35 more
  1. 2022Transformers learn in-context by gradient descentcited by 714
  2. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  3. 2021MetaICL: Learning to Learn In Contextcited by 450
  4. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  5. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  6. 2023In-Context Learning Creates Task Vectorscited by 302
  7. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  8. 2023Sentiment Analysis in the Era of Large Language Models: A Reality Checkcited by 262
  9. 2022Transformer Memory as a Differentiable Search Indexcited by 242
  10. 2024Many-Shot In-Context Learningcited by 203
  11. 2023Time Travel in LLMs: Tracing Data Contamination in Large Language Modelscited by 175
  12. 2022Large Language Models with Controllable Working Memorycited by 169
  13. 2017Machine Learning Models that Remember Too Muchcited by 165
  14. 2022A Review on Language Models as Knowledge Basescited by 156
  15. 2021Counterfactual Memorization in Neural Language Modelscited by 155
  16. 2023Emergent and Predictable Memorization in Large Language Modelscited by 147
  17. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  18. 2024In-Context Learning with Long-Context Models: An In-Depth Explorationcited by 126
  19. 2021How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVENcited by 124
  20. 2022Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrationscited by 116
  21. 2023Copyright Violations and Large Language Modelscited by 108
  22. 2017Ethical Challenges in Data-Driven Dialogue Systemscited by 86
  23. 2022Parallel Context Windows for Large Language Modelscited by 81
  24. 2022Understanding Transformer Memorization Recall Through Idiomscited by 62
  25. 2022Prototypical Calibration for Few-shot Learning of Language Modelscited by 58
  26. 2024From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examplescited by 55
  27. 2023Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Modelscited by 46
  28. 2024Dual Operating Modes of In-Context Learningcited by 42
  29. 2024Proving membership in LLM pretraining data via data watermarkscited by 38
  30. 2020Understanding Unintended Memorization in Federated Learningcited by 36
  31. 2024Copyright Traps for Large Language Modelscited by 36
  32. 2023Towards Understanding In-Context Learning with Contrastive Demonstrations and Saliency Mapscited by 35
  33. 2023In-Context Learning for Text Classification with Many Labelscited by 35
  34. 2023In-context Learning and Gradient Descent Revisitedcited by 20
  35. 2023Source Attribution for Large Language Model-Generated Datacited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 2.

6 papers.

  1. 2024ELICIT: LLM Augmentation via External In-Context Capabilitycited by 7
  2. 2025Can LLMs Judge Debates? Evaluating Non-Linear Reasoning via Argumentation Theory Semanticscited by 4
  3. 2025Towards Compute-Optimal Many-Shot In-Context Learningcited by 3
  4. 2025Illusion or Algorithm? Investigating Memorization, Emergence, and Symbolic Processing in In-Context Learningcited by 2
  5. 2026Intent Laundering: AI Safety Datasets Are Not What They Seemcited by 1
  6. 2026RA-CoA: Training-free Fashion Image Captioning via Retrieval-Augmented Chain-of-Attributescited by 0