mlsanityData to 7 Oct 2026

PoisonedParrot: Subtle Data Poisoning Attacks to Elicit Copyright-Infringing Content from Large Language Models

arXiv 2503.07697 ↗First posted 10 Mar 2025cs.LG cs.CRcited by 3

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

As the capabilities of large language models (LLMs) continue to expand, their usage has become increasingly prevalent. However, as reflected in numerous ongoing lawsuits regarding LLM-generated content, addressing copyright infringement remains a significant challenge. In this paper, we introduce PoisonedParrot: the first stealthy data poisoning attack that induces an LLM to generate copyrighted content even when the model has not been directly trained on the specific copyrighted material. PoisonedParrot integrates small fragments of copyrighted text into the poison samples using an off-the-shelf LLM. Despite its simplicity, evaluated in a wide range of experiments, PoisonedParrot is surprisingly effective at priming the model to generate copyrighted content with no discernible side effects. Moreover, we discover that existing defenses are largely ineffective against our attack. Finally, we make the first attempt at mitigating copyright-infringement poisoning attacks by proposing a defense: ParrotTrap. We encourage the community to explore this emerging threat model further.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 40 of its 57 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

40 of 57 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023PoisonPrompt: Backdoor Attack on Prompt-based Large Language Models19 Oct
  2. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection31 Jul
  3. 2022Not All Poisons are Created Equal: Robust Training against Data Poisoning18 Oct
  4. 2021Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models29 Mar

Other references · 36

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  4. 2016Deep Learning with Differential Privacycited by 2,682
  5. 2020Extracting Training Data from Large Language Modelscited by 2,144
  6. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  7. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  8. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  9. 2023Large Language Modelscited by 774
  10. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  11. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  12. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
24 more
  1. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  2. 2023Scaling Data-Constrained Language Modelscited by 391
  3. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  4. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  5. 2017Certified Defenses for Data Poisoning Attackscited by 288
  6. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
  7. 2023Poisoning Language Models During Instruction Tuningcited by 254
  8. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
  9. 2024Towards Safer Large Language Models through Machine Unlearningcited by 134
  10. 2021Large-Scale Differentially Private BERTcited by 117
  11. 2020RAB: Provable Robustness Against Backdoor Attackscited by 115
  12. 2020You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completioncited by 112
  13. 2023Copyright Violations and Large Language Modelscited by 108
  14. 2024LLM Dataset Inference: Did you train on my dataset?cited by 107
  15. 2022Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacycited by 97
  16. 2022Truth Serum: Poisoning Machine Learning Models to Reveal Their Secretscited by 94
  17. 2024Rethinking LLM Memorization through the Lens of Adversarial Compressioncited by 91
  18. 2024Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Datacited by 83
  19. 2018Technical Report: When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attackscited by 78
  20. 2024Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMscited by 50
  21. 2024The Stronger the Diffusion Model, the Easier the Backdoor: Data Poisoning to Induce Copyright Breaches Without Adjusting Finetuning Pipelinecited by 33
  22. 2022Just Fine-tune Twice: Selective Differential Privacy for Large Language Modelscited by 32
  23. 2024Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Modelscited by 28
  24. 2022Provably Confidential Language Modellingcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2026AttenMIA: LLM Membership Inference Attack through Attention Signalscited by 4
  2. 2025What Really is a Member? Discrediting Membership Inference via Poisoningcited by 2
  3. 2025ME: Trigger Element Combination Backdoor Attack on Copyright Infringementcited by 2