mlsanityData to 14 Oct 2026

Quantifying In-Context Reasoning Effects and Memorization Effects in LLMs

arXiv 2405.11880 ↗First posted 20 May 2024cs.LG cs.AI cs.CLcited by 3

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

In this study, we propose an axiomatic system to define and quantify the precise memorization and in-context reasoning effects used by the large language model (LLM) for language generation. These effects are formulated as non-linear interactions between tokens/words encoded by the LLM. Specifically, the axiomatic system enables us to categorize the memorization effects into foundational memorization effects and chaotic memorization effects, and further classify in-context reasoning effects into enhanced inference patterns, eliminated inference patterns, and reversed inference patterns. Besides, the decomposed effects satisfy the sparsity property and the universal matching property, which mathematically guarantee that the LLM's confidence score can be faithfully decomposed into the memorization effects and in-context reasoning effects. Experiments show that the clear disentanglement of memorization effects and in-context reasoning effects enables a straightforward examination of detailed inference patterns encoded by LLMs.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 28 of its 31 references that are in the map. First posted on arXiv 20 May 2024.

Builds on

28 of 31 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023LLaMA: Open and Efficient Foundation Language Models27 Feb

Other references · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  5. 2022Emergent Abilities of Large Language Modelscited by 2,512
  6. 2020Extracting Training Data from Large Language Modelscited by 2,189
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  8. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  9. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 807
  10. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  11. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  12. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
15 more
  1. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 497
  2. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 220
  3. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 217
  4. 2020Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learningcited by 216
  5. 2021Counterfactual Memorization in Neural Language Modelscited by 156
  6. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
  7. 2021How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVENcited by 124
  8. 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
  9. 2023Does a Neural Network Really Encode Symbolic Concepts?cited by 27
  10. 2023Explaining Generalization Power of a DNN Using Interactive Conceptscited by 25
  11. 2024Defining and Extracting generalizable interaction primitives from DNNscited by 14
  12. 2023Technical Note: Defining and Quantifying AND-OR Interactions for Faithful and Concise Explanation of DNNscited by 12
  13. 2023Can the Inference Logic of Large Language Models be Disentangled into Symbolic Concepts?cited by 11
  14. 2023Where We Have Arrived in Proving the Emergence of Sparse Symbolic Concepts in AI Modelscited by 8
  15. 2023Explaining How a Neural Network Play the Go Game and Let People Learncited by 6

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2025Diagnosing Memorization in Chain-of-Thought Reasoning, One Token at a Timecited by 4
  2. 2024Neuron-Level Differentiation of Memorization and Generalization in Large Language Modelscited by 2
  3. 2025Pitfalls of Scale: Investigating the Inverse Task of Redefinition in Large Language Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026MemoReason: Evaluating the Effect of Parametric Memory on Contextual Reasoning in LLMs
  2. 2024Disentangling Memory and Reasoning Ability in Large Language Models
  3. 2024On Memorization of Large Language Models in Logical Reasoning
  4. 2025The Reasoning-Memorization Interplay in Language Models Is Mediated by a Single Direction
  5. 2025Can an LLM Induce a Graph? Investigating Memory Drift and Context Length
  6. 2025Unveiling Over-Memorization in Finetuning LLMs for Reasoning Tasks
  7. 2025The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigation
  8. 2025Understanding Verbatim Memorization in LLMs Through Circuit Discovery