mlsanityData to 14 Oct 2026

Counterfactual Influence as a Distributional Quantity

arXiv 2506.20481 ↗First posted 25 Jun 2025cs.LG cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Machine learning models are known to memorize samples from their training data, raising concerns around privacy and generalization. Counterfactual self-influence is a popular metric to study memorization, quantifying how the model's prediction for a sample changes depending on the sample's inclusion in the training dataset. However, recent work has shown memorization to be affected by factors beyond self-influence, with other training samples, in particular (near-)duplicates, having a large impact. We here study memorization treating counterfactual influence as a distributional quantity, taking into account how all training samples influence how a sample is memorized. For a small language model, we compute the full influence distribution of training samples on each other and analyze its properties. We find that solely looking at self-influence can severely underestimate tangible risks associated with memorization: the presence of (near-)duplicates seriously reduces self-influence, while we find these samples to be (near-)extractable. We observe similar patterns for image classification, where simply looking at the influence distributions reveals the presence of near-duplicates in CIFAR-10. Our findings highlight that memorization stems from complex interactions across training data and is better captured by the full influence distribution than by self-influence alone.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 34 references that are in the map. First posted on arXiv 25 Jun 2025.

Builds on

28 of 34 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Extracting Training Data from Large Language Modelscited by 2,189
  3. 2019Language Models as Knowledge Bases?cited by 1,936
  4. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  5. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  6. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  7. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 580
  8. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 550
  9. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 497
  10. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  11. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  12. 2023Detecting Pretraining Data from Large Language Modelscited by 387
16 more
  1. 2023Studying Large Language Model Generalization with Influence Functionscited by 326
  2. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 314
  3. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 258
  4. 2023Proving Test Set Contamination in Black Box Language Modelscited by 230
  5. 2024Do Membership Inference Attacks Work on Large Language Models?cited by 195
  6. 2021Counterfactual Memorization in Neural Language Modelscited by 156
  7. 2023Copyright Violations and Large Language Modelscited by 108
  8. 2022Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacycited by 100
  9. 2024What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functionscited by 94
  10. 2024Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Datacited by 85
  11. 2023Did the Neurons Read your Book? Document-level Membership Inference for Large Language Modelscited by 56
  12. 2022SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learningcited by 51
  13. 2024Measuring memorization in language models via probabilistic extractioncited by 41
  14. 2024Copyright Traps for Large Language Modelscited by 37
  15. 2023MoPe: Model Perturbation-based Privacy Attacks on Language Modelscited by 25
  16. 2025Language Models May Verbatim Complete Text They Were Not Explicitly Trained Oncited by 21

Built on by

Not cited yet.