mlsanityData to 14 Oct 2026

Copyright-Protected Language Generation via Adaptive Model Fusion

arXiv 2412.06619 ↗First posted 9 Dec 2024cs.LG cs.CL cs.CRcited by 7

Not on the map. Why?

Abstract (arXiv)

The risk of language models reproducing copyrighted material from their training data has led to the development of various protective measures. Among these, inference-time strategies that impose constraints via post-processing have shown promise in addressing the complexities of copyright regulation. However, they often incur prohibitive computational costs or suffer from performance trade-offs. To overcome these limitations, we introduce Copyright-Protecting Model Fusion (CP-Fuse), a novel approach that combines models trained on disjoint sets of copyrighted material during inference. In particular, CP-Fuse adaptively aggregates the model outputs to minimize the reproduction of copyrighted content, adhering to a crucial balancing property that prevents the regurgitation of memorized data. Through extensive experiments, we show that CP-Fuse significantly reduces the reproduction of protected material without compromising the quality of text and code generation. Moreover, its post-hoc nature allows seamless integration with other protective measures, further enhancing copyright safeguards. Lastly, we show that CP-Fuse is robust against common techniques for extracting training data.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 64 of its 76 references that are in the map. First posted on arXiv 9 Dec 2024.

Builds on

64 of 76 references.

References · 64

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2021Program Synthesis with Large Language Modelscited by 3,637
  7. 2016Deep Learning with Differential Privacycited by 2,728
  8. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  9. 2020Extracting Training Data from Large Language Modelscited by 2,189
  10. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  11. 2018Hierarchical Neural Story Generationcited by 1,193
  12. 2019Machine Unlearningcited by 1,061
38 more
  1. 2021Measuring Coding Challenge Competence With APPScited by 902
  2. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 818
  3. 2023StarCoder: may the source be with you!cited by 799
  4. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  5. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  6. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  7. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
  8. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 504
  9. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 446
  10. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 441
  11. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 437
  12. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 424
  13. 2022Dataless Knowledge Fusion by Merging Weights of Language Modelscited by 387
  14. 2023Who's Harry Potter? Approximate Unlearning in LLMscited by 381
  15. 2023GPTScore: Evaluate as You Desirecited by 361
  16. 2022The Stack: 3 TB of permissively licensed source codecited by 359
  17. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 314
  18. 2024Rethinking Machine Unlearning for Large Language Modelscited by 257
  19. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 220
  20. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMscited by 215
  21. 2020Mixed-Privacy Forgetting in Deep Networkscited by 163
  22. 2021Counterfactual Memorization in Neural Language Modelscited by 156
  23. 2024An Adversarial Perspective on Machine Unlearning for AI Safetycited by 123
  24. 2021Large-Scale Differentially Private BERTcited by 118
  25. 2023Foundation Models and Fair Usecited by 116
  26. 2024Knowledge Fusion of Large Language Modelscited by 115
  27. 2023Copyright Violations and Large Language Modelscited by 108
  28. 2022Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacycited by 100
  29. 2023NEFTune: Noisy Embeddings Improve Instruction Finetuningcited by 100
  30. 2023On Provable Copyright Protection for Generative Modelscited by 98
  31. 2023SILO Language Models: Isolating Legal Risk In a Nonparametric Datastorecited by 71
  32. 2023Scaling Expert Language Models with Unsupervised Domain Discoverycited by 60
  33. 2023Did the Neurons Read your Book? Document-level Membership Inference for Large Language Modelscited by 56
  34. 2024Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimizationcited by 56
  35. 2023Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Taskscited by 53
  36. 2024Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMscited by 50
  37. 2023Talkin' 'Bout AI Generation: Copyright and the Generative-AI Supply Chaincited by 46
  38. 2023Fusing Models with Complementary Expertisecited by 46

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2026Sparse Autoencoders are Capable LLM Jailbreak Mitigatorscited by 6
  2. 2025SUV: Scalable Large Language Model Copyright Compliance with Regularized Selective Unlearningcited by 5
  3. 2025A Lightweight Method to Disrupt Memorized Sequences in LLMcited by 2
  4. 2025Consensus Sampling for Safer Generative AIcited by 2
  5. 2026Provably Protecting Fine-Tuned LLMs from Training Data Extraction while Preserving Utilitycited by 0
  6. 2026CopyShield: A Cross-Level Benchmark of Copyright Defenses in LLMscited by 0
  7. 2026What the Model Cannot See: Covert and Provable Text Invisibility in LLMscited by 0