# Fantastic Copyrighted Beasts and How (Not) to Generate Them

arXiv: https://arxiv.org/abs/2406.14526 · HTML: https://mlsanity.com/paper/2406.14526/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 54 references that are in the map.
- First posted on arXiv 20 Jun 2024.

## Builds on (most cited first, 30 of 34)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [Evaluating Text-to-Visual Generation with Image-to-Text Generation](https://mlsanity.com/paper/2404.01291/) (2024)
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [Detecting Pretraining Data from Large Language Models](https://mlsanity.com/paper/2310.16789/) (2023)
- [One Embedder, Any Task: Instruction-Finetuned Text Embeddings](https://mlsanity.com/paper/2212.09741/) (2022)
- [VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2303.08320/) (2023)
- [Ablating Concepts in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.13516/) (2023)
- [Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.17591/) (2023)
- [Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generation](https://mlsanity.com/paper/2402.17245/) (2024)
- [Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts](https://mlsanity.com/paper/2309.06135/) (2023)
- [What's In My Big Data?](https://mlsanity.com/paper/2310.20707/) (2023)
- [Foundation Models and Fair Use](https://mlsanity.com/paper/2303.15715/) (2023)
- [A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions](https://mlsanity.com/paper/2312.08578/) (2023)
- [On Provable Copyright Protection for Generative Models](https://mlsanity.com/paper/2302.10870/) (2023)
- [Effective Prompt Extraction from Language Models](https://mlsanity.com/paper/2307.06865/) (2023)
- [SILO Language Models: Isolating Legal Risk In a Nonparametric Datastore](https://mlsanity.com/paper/2308.04430/) (2023)
- [Get What You Want, Not What You Don't: Image Content Suppression for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2402.05375/) (2024)
- [CPR: Retrieval Augmented Generation for Copyright Protection](https://mlsanity.com/paper/2403.18920/) (2024)

## Built on by (most cited first, 24 of 24)

- [MUSE: Machine Unlearning Six-Way Evaluation for Language Models](https://mlsanity.com/paper/2407.06460/) (2024)
- [On Memorization of Large Language Models in Logical Reasoning](https://mlsanity.com/paper/2410.23123/) (2024)
- [Evaluating Copyright Takedown Methods for Language Models](https://mlsanity.com/paper/2406.18664/) (2024)
- [NeuralOS: Towards Simulating Operating Systems via Neural Generative Models](https://mlsanity.com/paper/2507.08800/) (2025)
- [Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy](https://mlsanity.com/paper/2410.09591/) (2024)
- [OVERT: A Benchmark for Over-Refusal Evaluation on Text-to-Image Models](https://mlsanity.com/paper/2505.21347/) (2025)
- [LLM Unlearning Without an Expert Curated Dataset](https://mlsanity.com/paper/2508.06595/) (2025)
- [Provable unlearning in topic modeling and downstream tasks](https://mlsanity.com/paper/2411.12600/) (2024)
- [Negative Token Merging: Image-based Adversarial Feature Guidance](https://mlsanity.com/paper/2412.01339/) (2024)
- [Semantic to Structure: Learning Structural Representations for Infringement Detection](https://mlsanity.com/paper/2502.07323/) (2025)
- [On Surjectivity of Neural Networks: Can you elicit any behavior from your model?](https://mlsanity.com/paper/2508.19445/) (2025)
- [Quantifying Cross-Modality Memorization in Vision-Language Models](https://mlsanity.com/paper/2506.05198/) (2025)
- [The Pitfalls of "Security by Obscurity" And What They Mean for Transparent AI](https://mlsanity.com/paper/2501.18669/) (2025)
- [Towards Evaluation for Real-World LLM Unlearning](https://mlsanity.com/paper/2508.01324/) (2025)
- [Guardians of Generation: Dynamic Inference-Time Copyright Shielding with Adaptive Guidance for AI Image Generation](https://mlsanity.com/paper/2503.16171/) (2025)
- [Safer Prompts: Reducing Risks from Memorization in Visual Generative AI](https://mlsanity.com/paper/2505.03338/) (2025)
- [Rethinking Robust Adversarial Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2510.27285/) (2025)
- [Agentic Evaluation of Copyright Law Compliance](https://mlsanity.com/paper/2607.21799/) (2026)
- [CARLoS: Retrieval via Concise Assessment Representation of LoRAs at Scale](https://mlsanity.com/paper/2512.08826/) (2025)
- [Copyright Infringement Risk Reduction via Chain-of-Thought and Task Instruction Prompting](https://mlsanity.com/paper/2512.15442/) (2025)
- [We Should Separate Memorization from Copyright](https://mlsanity.com/paper/2602.08632/) (2026)
- [A Concept is More Than a Word: Diversified Unlearning in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2603.18767/) (2026)
- [Variable-Length Audio Fingerprinting](https://mlsanity.com/paper/2603.23947/) (2026)
- [Evaluating Intellectual Property Guardrails of Generative Image Models: A Technical Report](https://mlsanity.com/paper/2607.02582/) (2026)

## Abstract (arXiv)

Recent studies show that image and video generation models can be prompted to reproduce copyrighted content from their training data, raising serious legal concerns about copyright infringement. Copyrighted characters (e.g., Mario, Batman) present a significant challenge: at least one lawsuit has already awarded damages based on the generation of such characters. Consequently, commercial services like DALL-E have started deploying interventions. However, little research has systematically examined these problems: (1) Can users easily prompt models to generate copyrighted characters, even if it is unintentional?; (2) How effective are the existing mitigation strategies? To address these questions, we introduce a novel evaluation framework with metrics that assess both the generated image's similarity to copyrighted characters and its consistency with user intent, grounded in a set of popular copyrighted characters from diverse studios and regions. We show that state-of-the-art image and video generation models can still generate characters even if characters' names are not explicitly mentioned, sometimes with only two generic keywords (e.g., prompting with "videogame, plumber" consistently generates Nintendo's Mario character). We also introduce semi-automatic techniques to identify such keywords or descriptions that trigger character generation. Using this framework, we evaluate mitigation strategies, including prompt rewriting and new approaches we propose. Our findings reveal that common methods, such as DALL-E's prompt rewriting, are insufficient alone and require supplementary strategies like negative prompting. Our work provides empirical grounding for discussions on copyright mitigation strategies and offers actionable insights for model deployers implementing these safeguards.