# Analyzing Memorization in Large Language Models through the Lens of Model Attribution

arXiv: https://arxiv.org/abs/2501.05078 · HTML: https://mlsanity.com/paper/2501.05078/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 9 Jan 2025.

## Builds on (most cited first, 19 of 19)

- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [Does Learning Require Memorization? A Short Tale about a Long Tail](https://mlsanity.com/paper/1906.05271/) (2019)
- [Scalable Extraction of Training Data from (Production) Language Models](https://mlsanity.com/paper/2311.17035/) (2023)
- [Physics of Language Models: Part 3.1, Knowledge Storage and Extraction](https://mlsanity.com/paper/2309.14316/) (2023)
- [Are Large Pre-Trained Language Models Leaking Your Personal Information?](https://mlsanity.com/paper/2205.12628/) (2022)
- [Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws](https://mlsanity.com/paper/2404.05405/) (2024)
- [Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process](https://mlsanity.com/paper/2407.20311/) (2024)
- [Do Language Models Plagiarize?](https://mlsanity.com/paper/2203.07618/) (2022)
- [Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon](https://mlsanity.com/paper/2406.17746/) (2024)
- [A deeper look at depth pruning of LLMs](https://mlsanity.com/paper/2407.16286/) (2024)
- [Quantifying Memorization and Detecting Training Data of Pre-trained Language Models using Japanese Newspaper](https://mlsanity.com/paper/2404.17143/) (2024)

## Built on by (most cited first, 14 of 14)

- [ReLearn: Unlearning via Learning for Large Language Models](https://mlsanity.com/paper/2502.11190/) (2025)
- [The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigation](https://mlsanity.com/paper/2507.05578/) (2025)
- [When Benchmarks Leak: Inference-Time Decontamination for LLMs](https://mlsanity.com/paper/2601.19334/) (2026)
- [Realizing LLMs' Causal Potential Requires Science-Grounded, Novel Benchmarks](https://mlsanity.com/paper/2510.16530/) (2025)
- [Auditing Language Model Unlearning via Information Decomposition](https://mlsanity.com/paper/2601.15111/) (2026)
- [From Memorization to Reasoning in the Spectrum of Loss Curvature](https://mlsanity.com/paper/2510.24256/) (2025)
- [Caliper: Probing Lexical Anchors versus Causal Structure in LLMs](https://mlsanity.com/paper/2606.04915/) (2026)
- [Evaluating book summaries from internal knowledge in Large Language Models: a cross-model and semantic consistency approach](https://mlsanity.com/paper/2503.21613/) (2025)
- [Guess or Recall? Training CNNs to Classify and Localize Memorization in LLMs](https://mlsanity.com/paper/2508.02573/) (2025)
- [CLaw: Benchmarking Chinese Legal Knowledge in Large Language Models - A Fine-grained Corpus and Reasoning Analysis](https://mlsanity.com/paper/2509.21208/) (2025)
- [LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs](https://mlsanity.com/paper/2606.06286/) (2026)
- [Beyond Frequency: The Role of Redundancy in Large Language Model Memorization](https://mlsanity.com/paper/2506.12321/) (2025)
- [Metaphor Is Not All Attention Needs](https://mlsanity.com/paper/2605.12128/) (2026)
- [Where Grokking Happens: Distributed Utility and Fourier Recoding Without a Module Switch](https://mlsanity.com/paper/2609.17571/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are prevalent in modern applications but often memorize training data, leading to privacy breaches and copyright issues. Existing research has mainly focused on posthoc analyses, such as extracting memorized content or developing memorization metrics, without exploring the underlying architectural factors that contribute to memorization. In this work, we investigate memorization from an architectural lens by analyzing how attention modules at different layers impact its memorization and generalization performance. Using attribution techniques, we systematically intervene in the LLM architecture by bypassing attention modules at specific blocks while keeping other components like layer normalization and MLP transformations intact. We provide theorems analyzing our intervention mechanism from a mathematical view, bounding the difference in layer outputs with and without our attributions. Our theoretical and empirical analyses reveal that attention modules in deeper transformer blocks are primarily responsible for memorization, whereas earlier blocks are crucial for the models generalization and reasoning capabilities. We validate our findings through comprehensive experiments on different LLM families (Pythia and GPTNeo) and five benchmark datasets. Our insights offer a practical approach to mitigate memorization in LLMs while preserving their performance, contributing to safer and more ethical deployment in real world applications.