mlsanityData to 7 Oct 2026

Copyright Violations and Large Language Models

arXiv 2310.13771 ↗First posted 20 Oct 2023cs.CL cs.AIcited by 108

Not on the map. Why?

Cited by 57 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Language models may memorize more than just facts, including entire chunks of texts seen during training. Fair use exemptions to copyright laws typically allow for limited use of copyrighted material without permission from the copyright holder, but typically for extraction of information from copyrighted materials, rather than em verbatim reproduction. This work explores the issue of copyright violations and large language models through the lens of verbatim memorization, focusing on possible redistribution of copyrighted text. We present experiments with a range of language models over a collection of popular books and coding problems, providing a conservative characterization of the extent to which language models can redistribute these materials. Overall, this research highlights the need for further examination and the potential impact on future developments in natural language processing to ensure adherence to copyright regulations. Code is at urlhttps://github.com/coastalcph/CopyrightLLMs.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Builds on 15 of its 23 references that are in the map. First posted on arXiv 20 Oct 2023.

Builds on

15 of 23 references.

References · 15

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  5. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  6. 2020Extracting Training Data from Large Language Modelscited by 2,144
  7. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  8. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  9. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  10. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  11. 2020The Pushshift Reddit Datasetcited by 289
  12. 2018A Simple Method for Commonsense Reasoningcited by 223
3 more
  1. 2021On the geometry of generalization and memorization in deep neural networkscited by 71
  2. 2021Memorization vs. Generalization: Quantifying Data Leakage in NLP Performance Evaluationcited by 69
  3. 2023Controlling the Extraction of Memorized Data from Large Language Models via Prompt-Tuningcited by 40

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 38; 2025: 47; 2026: 22.

108 papers.

Sort:

By area:

  1. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  2. 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 153
  3. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  4. 2024On Memorization of Large Language Models in Logical Reasoningcited by 114
  5. 2024An Adversarial Perspective on Machine Unlearning for AI Safetycited by 113
  6. 2024Large Language Model Unlearning via Embedding-Corrupted Promptscited by 112
  7. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  8. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 100
  9. 2025MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbationscited by 88
  10. 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metricscited by 80
  11. 2024RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Modelscited by 77
  12. 2024DE-COP: Detecting Copyrighted Content in Language Models Training Datacited by 72
38 more
  1. 2024LLM Unlearning via Loss Adjustment with Only Forget Datacited by 71
  2. 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)cited by 54
  3. 2024Evaluating Copyright Takedown Methods for Language Modelscited by 42
  4. 2025Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMscited by 41
  5. 2025Exploring Criteria of Loss Reweighting to Enhance LLM Unlearningcited by 39
  6. 2024Offset Unlearning for Large Language Modelscited by 33
  7. 2024Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenoncited by 30
  8. 2024Navigating LLM Ethics: Advancements, Challenges, and Future Directionscited by 28
  9. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 24
  10. 2025BLUR: A Bi-Level Optimization Approach for LLM Unlearningcited by 23
  11. 2024The Frontier of Data Erasure: Machine Unlearning for Large Language Modelscited by 17
  12. 2025GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detectioncited by 17
  13. 2025Large Language Models for Computer-Aided Design: A Surveycited by 16
  14. 2024Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data?cited by 15
  15. 2024SoK: Dataset Copyright Auditing in Machine Learning Systemscited by 14
  16. 2024LiCoEval: Evaluating LLMs on License Compliance in Code Generationcited by 13
  17. 2024Privacy-Aware Visual Language Modelscited by 12
  18. 2024Mitigating Memorization In Language Modelscited by 11
  19. 2025Blackbox Model Provenance via Palimpsestic Membership Inferencecited by 11
  20. 2024Developing Story: Case Studies of Generative AI's Use in Journalismcited by 10
  21. 2024The Mosaic Memory of Large Language Modelscited by 9
  22. 2024Secret Use of Large Language Model (LLM)cited by 9
  23. 2025Agents Are All You Need for LLM Unlearningcited by 9
  24. 2025UCD: Unlearning in LLMs via Contrastive Decodingcited by 9
  25. 2025LLMs for Customized Marketing Content Generation and Evaluation at Scalecited by 9
  26. 2025The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigationcited by 9
  27. 2025Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMscited by 9
  28. 2025Leak@k: Unlearning Does Not Make LLMs Forget Under Probabilistic Decodingcited by 9
  29. 2024Towards More Realistic Extraction Attacks: An Adversarial Perspectivecited by 8
  30. 2025Discovering Forbidden Topics in Language Modelscited by 8
  31. 2025On The Fragility of Benchmark Contamination Detection in Reasoning Modelscited by 8
  32. 2025The Surprising Effectiveness of Membership Inference with Simple N-Gram Coveragecited by 7
  33. 2025Extracting alignment data in open modelscited by 7
  34. 2024A Little Leak Will Sink a Great Ship: Survey of Transparency for Large Language Models from Start to Finishcited by 6
  35. 2024Memorization in In-Context Learningcited by 6
  36. 2024The Unreasonable Ineffectiveness of Nucleus Sampling on Mitigating Text Memorizationcited by 6
  37. 2024Copyright-Protected Language Generation via Adaptive Model Fusioncited by 6
  38. 2025Improving LLM Unlearning Robustness via Random Perturbationscited by 6

50 most cited of 108 All 108 (JSON)