mlsanityData to 13 Oct 2026

Pruning as a Defense: Reducing Memorization in Large Language Models

arXiv 2502.15796 ↗First posted 18 Feb 2025cs.LG cs.AI cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Large language models have been shown to memorize significant portions of their training data, which they can reproduce when appropriately prompted. This work investigates the impact of simple pruning techniques on this behavior. Our findings reveal that pruning effectively reduces the extent of memorization in LLMs, demonstrating its potential as a foundational approach for mitigating membership inference attacks.

Key facts. Cited by 3 papers in the map (data through 13 Oct 2026). Builds on 8 of its 10 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

8 of 10 references.

References · 8

  1. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,379
  2. 2020Extracting Training Data from Large Language Modelscited by 2,181
  3. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  4. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 263
  5. 2024The Unreasonable Ineffectiveness of the Deeper Layerscited by 221
  6. 2023Task-Specific Skill Localization in Fine-tuned Language Modelscited by 105
  7. 2024Dissecting Language Models: Machine Unlearning via Selective Pruningcited by 41
  8. 2023Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarizationcited by 16

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Mitigating Memorization Where It Happenscited by 2
  2. 2026What Hard Tokens Reveal: Exploiting Low-confidence Tokens for Membership Inference Attacks against Large Language Modelscited by 0
  3. 2026Bits and Memories: Measuring Verbatim Extraction Across LLM Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025PerProb: Indirectly Evaluating Memorization in Large Language Models ↗
  2. 2025Fewer Weights, More Problems: A Practical Attack on LLM Pruning
  3. 2025The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigation
  4. 2025Pruning Laws for Large Language Models
  5. 2024Undesirable Memorization in Large Language Models: A Survey
  6. 2024Mitigating Memorization In Language Models
  7. 2025Analyzing Memorization in Large Language Models through the Lens of Model Attribution
  8. 2024Context-Aware Membership Inference Attacks against Pre-trained Large Language Models