mlsanityData to 7 Oct 2026

Memorization in Language Models through the Lens of Intrinsic Dimension

arXiv 2506.09591 ↗First posted 11 Jun 2025cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Language Models (LMs) are prone to memorizing parts of their data during training and unintentionally emitting them at generation time, raising concerns about privacy leakage and disclosure of intellectual property. While previous research has identified properties such as context length, parameter size, and duplication frequency, as key drivers of unintended memorization, little is known about how the latent structure modulates this rate of memorization. We investigate the role of Intrinsic Dimension (ID), a geometric proxy for the structural complexity of a sequence in latent space, in modulating memorization. Our findings suggest that ID acts as a suppressive signal for memorization: compared to low-ID sequences, high-ID sequences are less likely to be memorized, particularly in overparameterized models and under sparse exposure. These findings highlight the interaction between scale, exposure, and complexity in shaping memorization.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 24 of its 37 references that are in the map. First posted on arXiv 11 Jun 2025.

Builds on

24 of 37 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  6. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  7. 2020Extracting Training Data from Large Language Modelscited by 2,144
  8. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  9. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  10. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  11. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  12. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
12 more
  1. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  2. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  3. 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 307
  4. 2019Intrinsic dimension of data representations in deep neural networkscited by 276
  5. 2018Estimating the intrinsic dimension of datasets by a minimal neighborhood informationcited by 246
  6. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 195
  7. 2023Intrinsic Dimension Estimation for Robust Detection of AI-Generated Textscited by 111
  8. 2020When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?cited by 102
  9. 2022Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacycited by 97
  10. 2024Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Datacited by 83
  11. 2021Intrinsic Dimension, Persistent Homology and Generalization in Neural Networkscited by 70
  12. 2018Persistent Homology and the Upper Box Dimensioncited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggerscited by 13
  2. 2025Unveiling Intrinsic Dimension of Texts: from Academic Abstract to Creative Storycited by 1
  3. 2026A Comparative Analysis of LLM Memorization at Statistical and Internal Levels: Cross-Model Commonalities and Model-Specific Signaturescited by 0