Data to 7 Oct 2026

Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs

arXiv 2406.10209 ↗First posted 14 Jun 2024cs.CLcited by 50

Not on the map. Why?

Abstract (arXiv)

Large language models can memorize and repeat their training data, causing privacy and copyright risks. To mitigate memorization, we introduce a subtle modification to the next-token training objective that we call the goldfish loss. During training, randomly sampled subsets of tokens are excluded from the loss computation. These dropped tokens are not memorized by the model, which prevents verbatim reproduction of a complete chain of tokens from the training set. We run extensive experiments training billion-scale Llama-2 models, both pre-trained and trained from scratch, and demonstrate significant reductions in extractable memorization with little to no impact on downstream benchmarks.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 30 of its 45 references that are in the map. First posted on arXiv 14 Jun 2024.

Builds on

30 of 45 references.

References · 30

2020Language Models are Few-Shot Learnerscited by 32,199
2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
2016Deep Learning with Differential Privacycited by 2,682
2020Extracting Training Data from Large Language Modelscited by 2,144
2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
2022Quantifying Memorization Across Neural Language Modelscited by 755
2021Deduplicating Training Data Makes Language Models Bettercited by 705
2023A Watermark for Large Language Modelscited by 667
2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 484
2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466

18 more

2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 424
2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
2023Who's Harry Potter? Approximate Unlearning in LLMscited by 366
2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
2024Rho-1: Not All Tokens Are What You Needcited by 132
2021Large-Scale Differentially Private BERTcited by 117
2023Ambient Diffusion: Learning Clean Distributions from Corrupted Datacited by 108
2023NEFTune: Noisy Embeddings Improve Instruction Finetuningcited by 99
2024Rethinking LLM Memorization through the Lens of Adversarial Compressioncited by 91
2022Do Language Models Plagiarize?cited by 72
2024Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Datacited by 44
2022Token Dropping for Efficient BERT Pretrainingcited by 42
2022Just Fine-tune Twice: Selective Differential Privacy for Large Language Modelscited by 32
2024Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Modelscited by 28
2021AxoNN: An asynchronous, message-driven parallel framework for extreme-scale deep learningcited by 15
2022Provably Confidential Language Modellingcited by 13
2023A 4D Hybrid Algorithm to Scale Parallel Training to Thousands of GPUscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 30; 2026: 10.

50 papers.

Sort:

By area:

2024Demystifying Verbatim Memorization in Large Language Modelscited by 55
2024Evaluating Copyright Takedown Methods for Language Modelscited by 42
2024Undesirable Memorization in Large Language Models: A Surveycited by 29
2024What Do Learning Dynamics Reveal About Generalization in LLM Reasoning?cited by 22
2025Language Models May Verbatim Complete Text They Were Not Explicitly Trained Oncited by 20
2024Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data?cited by 15
2025Tokens for Learning, Tokens for Unlearning: Mitigating Membership Inference Attacks in Large Language Models via Dual-Purpose Trainingcited by 14
2024Mitigating Memorization In Language Modelscited by 11
2025Unveiling Over-Memorization in Finetuning LLMs for Reasoning Taskscited by 10
2024Creativity in AI: Progresses and Challengescited by 7
2024Understanding and Mitigating Memorization in Diffusion Models for Tabular Datacited by 7
2025Democratizing AI: Open-source Scalable LLM Training on GPU-based Supercomputerscited by 7

38 more

2025Mitigating Memorization in LLMs using Activation Steeringcited by 7
2025Redirection for Erasing Memory (REM): Towards a universal unlearning method for corrupted datacited by 7
2024Copyright-Protected Language Generation via Adaptive Model Fusioncited by 6
2025Mitigating Unintended Memorization with LoRA in Federated Learning for LLMscited by 6
2025ParaPO: Aligning Language Models to Reduce Verbatim Reproduction of Pre-training Datacited by 6
2025PANORAMA: A synthetic PII-laced dataset for studying sensitive data memorization in LLMscited by 6
2025Incident Analysis for AI Agentscited by 5
2024Strong Copyright Protection for Language Models via Adaptive Model Fusioncited by 4
2025Obliviate: Efficient Unmemorization for Protecting Intellectual Property in Large Language Modelscited by 4
2025SUV: Scalable Large Language Model Copyright Compliance with Regularized Selective Unlearningcited by 4
2025Generative AI Training and Copyright Lawcited by 3
2025PoisonedParrot: Subtle Data Poisoning Attacks to Elicit Copyright-Infringing Content from Large Language Modelscited by 3
2025LayerCake: Token-Aware Contrastive Decoding within Large Language Model Layerscited by 3
2025Can Performant LLMs Be Ethical? Quantifying the Impact of Web Crawling Opt-Outscited by 2
2025Teaching Models to Understand (but not Generate) High-risk Datacited by 2
2025Ready2Unlearn: A Learning-Time Approach for Preparing Models with Future Unlearning Readinesscited by 2
2025Positional Fragility in LLMs: How Offset Effects Reshape Our Understanding of Memorization Riskscited by 2
2026PrivacyAlign: Contextual Privacy Alignment for LLM Agentscited by 2
2025A Lightweight Method to Disrupt Memorized Sequences in LLMcited by 1
2025DIS-CO: Discovering Copyrighted Content in VLMs Training Datacited by 1
2025Leaking LoRa: An Evaluation of Password Leaks and Knowledge Storage in Large Language Modelscited by 1
2025A Closer Look on Memorization in Tabular Diffusion Model: A Data-Centric Perspectivecited by 1
2025Win-k: Improved Membership Inference Attacks on Small Language Modelscited by 1
2025MultiMat: Multimodal Program Synthesis for Procedural Materials using Large Multimodal Modelscited by 1
2026Segment-Level Attribution for Selective Learning of Long Reasoning Tracescited by 1
2026Beyond Indistinguishability: Measuring Extraction Risk in LLM APIscited by 1
2026NumLeak: Public Numeric Benchmarks as Latent Labels in Foundation Modelscited by 1
2025Certified Mitigation of Worst-Case LLM Copyright Infringementcited by 0
2025The Synthetic Mirror -- Synthetic Data at the Age of Agentic AIcited by 0
2025Early Detection and Reduction of Memorisation for Domain Adaptation and Instruction Tuningcited by 0
2025RECAP: Reproducing Copyrighted Data from LLMs Training with an Agentic Pipelinecited by 0
2025In-Context Learning Without Copyingcited by 0
2026Provably Protecting Fine-Tuned LLMs from Training Data Extraction while Preserving Utilitycited by 0
2026Output Vector Editing for Memorization Mitigation in Large Language Modelscited by 0
2026Anchored or Drifting: What Recursive Self-Generation Reveals About Training Datacited by 0
2026Crossing the Margin Cliff: Toward Relearn-Robust LLM Unlearning via Margin Calibrationcited by 0
2026The Privacy-Hallucination Tradeoff in Differentially Private Language Modelscited by 0
2026Rebalancing Token Importance in Language Models with TF-IDF Weighted Cross-Entropy Losscited by 0