# Causal Estimation of Memorisation Profiles

arXiv: https://arxiv.org/abs/2406.04327 · HTML: https://mlsanity.com/paper/2406.04327/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 59 references that are in the map.
- First posted on arXiv 6 Jun 2024.

## Builds on (most cited first, 30 of 37)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation](https://mlsanity.com/paper/2008.03703/) (2020)
- [Does Learning Require Memorization? A Short Tale about a Long Tail](https://mlsanity.com/paper/1906.05271/) (2019)
- [Dynabench: Rethinking Benchmarking in NLP](https://mlsanity.com/paper/2104.14337/) (2021)
- [Deduplicating Training Data Mitigates Privacy Risks in Language Models](https://mlsanity.com/paper/2202.06539/) (2022)
- [Influence Functions in Deep Learning Are Fragile](https://mlsanity.com/paper/2006.14651/) (2020)
- [If Influence Functions are the Answer, Then What is the Question?](https://mlsanity.com/paper/2209.05364/) (2022)
- [Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster](https://mlsanity.com/paper/2304.03208/) (2023)
- [On Provable Copyright Protection for Generative Models](https://mlsanity.com/paper/2302.10870/) (2023)
- [What Language Model to Train if You Have One Million GPU Hours?](https://mlsanity.com/paper/2210.15424/) (2022)
- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)
- [Do Language Models Plagiarize?](https://mlsanity.com/paper/2203.07618/) (2022)
- [Can Neural Network Memorization Be Localized?](https://mlsanity.com/paper/2307.09542/) (2023)
- [Measuring the Effect of Training Data on Deep Learning Predictions via Randomized Experiments](https://mlsanity.com/paper/2206.10013/) (2022)
- [Training Data Extraction From Pre-trained Language Models: A Survey](https://mlsanity.com/paper/2305.16157/) (2023)
- [Localizing Paragraph Memorization in Language Models](https://mlsanity.com/paper/2403.19851/) (2024)
- [Theoretical and Practical Perspectives on what Influence Functions Do](https://mlsanity.com/paper/2305.16971/) (2023)
- [Estimating Individual Treatment Effect in Observational Data Using Random Forest Methods](https://mlsanity.com/paper/1701.05306/) (2017)

## Built on by (most cited first, 20 of 20)

- [PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs](https://mlsanity.com/paper/2503.09543/) (2025)
- [Beyond Math: Stories as a Testbed for Memorization-Constrained Reasoning in LLMs](https://mlsanity.com/paper/2412.14368/) (2024)
- [To model human linguistic prediction, make LLMs less superhuman](https://mlsanity.com/paper/2510.05141/) (2025)
- [Blackbox Model Provenance via Palimpsestic Membership Inference](https://mlsanity.com/paper/2510.19796/) (2025)
- [Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention](https://mlsanity.com/paper/2605.29548/) (2026)
- [Free Record-Level Privacy Risk Evaluation Through Artifact-Based Methods](https://mlsanity.com/paper/2411.05743/) (2024)
- [Internal Data Repetition Destroys Language Models](https://mlsanity.com/paper/2606.24998/) (2026)
- [Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamics](https://mlsanity.com/paper/2606.06533/) (2026)
- [Bergson: An Open Source Library for Data Attribution](https://mlsanity.com/paper/2606.11660/) (2026)
- [Positional Fragility in LLMs: How Offset Effects Reshape Our Understanding of Memorization Risks](https://mlsanity.com/paper/2505.13171/) (2025)
- [Who Gets Credit or Blame? Attributing Accountability in Modern AI Systems](https://mlsanity.com/paper/2506.00175/) (2025)
- [Predicting Training Re-evaluation Curves Enables Effective Data Curriculums for LLMs](https://mlsanity.com/paper/2509.25380/) (2025)
- [A Prior-Aware Metric for Efficiently Distinguishing Memorization from Generalization in Large Language Models](https://mlsanity.com/paper/2602.18733/) (2026)
- [How Open Must Language Models be to Enable Reliable Scientific Inference?](https://mlsanity.com/paper/2603.26539/) (2026)
- [Applied Federated Model Personalisation in the Industrial Domain: A Comparative Study](https://mlsanity.com/paper/2409.06904/) (2024)
- [Estimating the Effects of Sample Training Orders for Large Language Models without Retraining](https://mlsanity.com/paper/2505.22042/) (2025)
- [Beyond Frequency: The Role of Redundancy in Large Language Model Memorization](https://mlsanity.com/paper/2506.12321/) (2025)
- [Linear Causal Representation Learning by Topological Ordering, Pruning, and Disentanglement](https://mlsanity.com/paper/2509.22553/) (2025)
- [Validity Threats for Foundation Model Research](https://mlsanity.com/paper/2606.05029/) (2026)
- [What Eviction Destroys: A Restore-Counterfactual Audit of Forgetting in Agent Memory](https://mlsanity.com/paper/2609.08279/) (2026)

## Abstract (arXiv)

Understanding memorisation in language models has practical and societal implications, e.g., studying models' training dynamics or preventing copyright infringements. Prior work defines memorisation as the causal effect of training with an instance on the model's ability to predict that instance. This definition relies on a counterfactual: the ability to observe what would have happened had the model not seen that instance. Existing methods struggle to provide computationally efficient and accurate estimates of this counterfactual. Further, they often estimate memorisation for a model architecture rather than for a specific model instance. This paper fills an important gap in the literature, proposing a new, principled, and efficient method to estimate memorisation based on the difference-in-differences design from econometrics. Using this method, we characterise a model's memorisation profile--its memorisation trends across training--by only observing its behaviour on a small set of instances throughout training. In experiments with the Pythia model suite, we find that memorisation (i) is stronger and more persistent in larger models, (ii) is determined by data order and learning rate, and (iii) has stable trends across model sizes, thus making memorisation in larger models predictable from smaller ones.