mlsanityData to 14 Oct 2026

Illusion or Algorithm? Investigating Memorization, Emergence, and Symbolic Processing in In-Context Learning

arXiv 2505.11004 ↗First posted 16 May 2025cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Large-scale Transformer language models (LMs) trained solely on next-token prediction with web-scale data can solve a wide range of tasks after seeing just a few examples. The mechanism behind this capability, known as in-context learning (ICL), remains both controversial and poorly understood. Some studies argue that it is merely the result of memorizing vast amounts of data, while others contend that it reflects a fundamental, symbolic algorithmic development in LMs. In this work, we introduce a suite of investigative tasks and a novel method to systematically investigate ICL by leveraging the full Pythia scaling suite, including interim checkpoints that capture progressively larger amount of training data. By carefully exploring ICL performance on downstream tasks and simultaneously conducting a mechanistic analysis of the residual stream's subspace, we demonstrate that ICL extends beyond mere "memorization" of the training corpus, yet does not amount to the implementation of an independent symbolic algorithm. Our results also clarify several aspects of ICL, including the influence of training dynamics, model capabilities, and elements of mechanistic interpretability. Overall, our work advances the understanding of ICL and its implications, offering model developers insights into potential improvements and providing AI security practitioners with a basis for more informed guidelines.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 42 of its 58 references that are in the map. First posted on arXiv 16 May 2025.

Builds on

42 of 58 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  3. 2022Emergent Abilities of Large Language Modelscited by 2,512
  4. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  5. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  6. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  7. 2022In-context Learning and Induction Headscited by 964
  8. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  9. 2022Transformers learn in-context by gradient descentcited by 732
  10. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  11. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 684
  12. 2024OLMo: Accelerating the Science of Language Modelscited by 613
30 more
  1. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 550
  2. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 523
  3. 2023Larger language models do in-context learning differentlycited by 394
  4. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
  5. 2022Data Distributional Properties Drive Emergent In-Context Learning in Transformerscited by 309
  6. 2023Function Vectors in Large Language Modelscited by 308
  7. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  8. 2022Predictability and Surprise in Large Generative Modelscited by 210
  9. 2022Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizerscited by 204
  10. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 200
  11. 2023Time Travel in LLMs: Tracing Data Contamination in Large Language Modelscited by 177
  12. 2024The Evolution of Statistical Induction Heads: In-Context Learning Markov Chainscited by 118
  13. 2023Are Emergent Abilities in Large Language Models just In-Context Learning?cited by 109
  14. 2023Tracr: Compiled Transformers as a Laboratory for Interpretabilitycited by 104
  15. 2023Task Contamination: Language Models May Not Be Few-Shot Anymorecited by 99
  16. 2024In-Context Language Learning: Architectures and Algorithmscited by 89
  17. 2023The Transient Nature of Emergent In-Context Learning in Transformerscited by 76
  18. 2025Which Attention Heads Matter for In-Context Learning?cited by 61
  19. 2024Competition Dynamics Shape Algorithmic Phases of In-Context Learningcited by 54
  20. 2023An Open Source Data Contamination Report for Large Language Modelscited by 51
  21. 2024Why Larger Language Models Do In-context Learning Differently?cited by 45
  22. 2024The broader spectrum of in-context learningcited by 35
  23. 2025Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Modelscited by 31
  24. 2023In-context Learning and Gradient Descent Revisitedcited by 20
  25. 2023Generative Models as a Complex Systems Science: How can we make sense of large language model behavior?cited by 19
  26. 2024Toward Understanding In-context vs. In-weight Learningcited by 19
  27. 2024Revisiting In-context Learning Inference Circuit in Large Language Modelscited by 17
  28. 2024Investigating the Pre-Training Dynamics of In-Context Learning: Task Recognition vs. Task Learningcited by 6
  29. 2024Memorization in In-Context Learningcited by 6
  30. 2024Mechanistic Behavior Editing of Language Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effectscited by 1
  2. 2025Neither Stochastic Parroting nor AGI: LLMs Solve Tasks through Context-Directed Extrapolation from Training Data Priorscited by 0
  3. 2026From Copying to Composition: How Language Models Develop In-Context Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Concept-aware Training Improves In-context Learning Ability of Language Models
  2. 2024Context-Scaling versus Task-Scaling in In-Context Learning
  3. 2025Probing In-Context Learning: Impact of Task Complexity and Model Architecture on Generalization and Efficiency
  4. 2026Dissecting Multimodal In-Context Learning: Modality Asymmetries and Circuit Dynamics in modern Transformers
  5. 2023Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability
  6. 2026Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
  7. 2023The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis
  8. 2025From Compression to Expression: A Layerwise Analysis of In-Context Learning