mlsanityData to 14 Oct 2026

The Unreasonable Ineffectiveness of Nucleus Sampling on Mitigating Text Memorization

arXiv 2408.16345 ↗First posted 29 Aug 2024cs.CLcited by 6

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

This work analyses the text memorization behavior of large language models (LLMs) when subjected to nucleus sampling. Stochastic decoding methods like nucleus sampling are typically applied to overcome issues such as monotonous and repetitive text generation, which are often observed with maximization-based decoding techniques. We hypothesize that nucleus sampling might also reduce the occurrence of memorization patterns, because it could lead to the selection of tokens outside the memorized sequence. To test this hypothesis we create a diagnostic dataset with a known distribution of duplicates that gives us some control over the likelihood of memorization of certain parts of the training data. Our analysis of two GPT-Neo models fine-tuned on this dataset interestingly shows that (i) an increase of the nucleus size reduces memorization only modestly, and (ii) even when models do not engage in "hard" memorization -- a verbatim reproduction of training samples -- they may still display "soft" memorization whereby they generate outputs that echo the training data but without a complete one-by-one resemblance.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 23 of its 33 references that are in the map. First posted on arXiv 29 Aug 2024.

Builds on

23 of 33 references.

References · 23

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  4. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  5. 2020Extracting Training Data from Large Language Modelscited by 2,189
  6. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  7. 2021Membership Inference Attacks From First Principlescited by 878
  8. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
  9. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  10. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 314
  11. 2017OpenNMT: Open-source Toolkit for Neural Machine Translationcited by 251
  12. 2022Data Contamination: From Memorization to Exploitationcited by 236
11 more
  1. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 200
  2. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
  3. 2022Scaling Laws and Interpretability of Learning from Repeated Datacited by 152
  4. 2017Controlling Linguistic Style Aspects in Neural Language Generationcited by 149
  5. 2021How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVENcited by 124
  6. 2023Copyright Violations and Large Language Modelscited by 108
  7. 2022Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacycited by 100
  8. 2022Do Language Models Plagiarize?cited by 73
  9. 2022Understanding Transformer Memorization Recall Through Idiomscited by 62
  10. 2018Importance of Search and Evaluation Strategies in Neural Dialogue Modelingcited by 44
  11. 2023Data Portraits: Recording Foundation Model Training Datacited by 30

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 2.

6 papers.

  1. 2024Undesirable Memorization in Large Language Models: A Surveycited by 29
  2. 2025Extracting alignment data in open modelscited by 8
  3. 2026Learning Adaptive LLM Decodingcited by 3
  4. 2025Bits Leaked per Query: Information-Theoretic Bounds on Adversarial Attacks against LLMscited by 1
  5. 2025GaLLoP: Gradient-based Sparse Learning on Low-Magnitude Parameterscited by 1
  6. 2026LLM-Metrics: Measuring Research Impact Through Large Language Model Memorycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Mitigating Memorization In Language Models
  2. 2026Characterizing Memorization in Diffusion Language Models: Generalized Extraction and Sampling Effects
  3. 2024Demystifying Verbatim Memorization in Large Language Models
  4. 2024Detecting, Explaining, and Mitigating Memorization in Diffusion Models
  5. 2025Understanding Verbatim Memorization in LLMs Through Circuit Discovery
  6. 2025Hubble: a Model Suite to Advance the Study of LLM Memorization
  7. 2025The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigation
  8. 2024Rethinking LLM Memorization through the Lens of Adversarial Compression