mlsanityData to 7 Oct 2026

DIS-CO: Discovering Copyrighted Content in VLMs Training Data

arXiv 2502.17358 ↗First posted 24 Feb 2025cs.CV cs.AI cs.LGcited by 1

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

How can we verify whether copyrighted content was used to train a large vision-language model (VLM) without direct access to its training data? Motivated by the hypothesis that a VLM is able to recognize images from its training corpus, we propose DIS-CO, a novel approach to infer the inclusion of copyrighted content during the model's development. By repeatedly querying a VLM with specific frames from targeted copyrighted material, DIS-CO extracts the content's identity through free-form text completions. To assess its effectiveness, we introduce MovieTection, a benchmark comprising 14,000 frames paired with detailed captions, drawn from films released both before and after a model's training cutoff. Our results show that DIS-CO significantly improves detection performance, nearly doubling the average AUC of the best prior method on models with logits available. Our findings also highlight a broader concern: all tested models appear to have been exposed to some extent to copyrighted content. Our code and data are available at https://github.com/avduarte333/DIS-CO

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 44 references that are in the map. First posted on arXiv 24 Feb 2025.

Builds on

35 of 44 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Membership Inference Attacks against Large Vision-Language Models5 Nov
  2. 2024Extracting Training Data from Document-Based VQA Models11 Jul

Other references · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2024GPT-4o System Cardcited by 4,467
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  8. 2020Extracting Training Data from Large Language Modelscited by 2,144
  9. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  10. 2021Membership Inference Attacks From First Principlescited by 839
  11. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  12. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
21 more
  1. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
  2. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  3. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
  4. 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 273
  5. 2024Do Membership Inference Attacks Work on Large Language Models?cited by 188
  6. 2024Pixtral 12Bcited by 174
  7. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 168
  8. 2021On the Importance of Difficulty Calibration in Membership Inference Attackscited by 137
  9. 2023Copyright Violations and Large Language Modelscited by 108
  10. 2024LLM Dataset Inference: Did you train on my dataset?cited by 107
  11. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 102
  12. 2018Understanding Membership Inferences on Well-Generalized Learning Modelscited by 83
  13. 2024DE-COP: Detecting Copyrighted Content in Language Models Training Datacited by 72
  14. 2024Blind Baselines Beat Membership Inference Attacks for Foundation Modelscited by 65
  15. 2024Pretraining Data Detection for Large Language Models: A Divergence-based Calibration Methodcited by 54
  16. 2023Did the Neurons Read your Book? Document-level Membership Inference for Large Language Modelscited by 51
  17. 2024Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMscited by 50
  18. 2024Membership Inference Attacks Cannot Prove that a Model Was Trained On Your Datacited by 49
  19. 2024Copyright Traps for Large Language Modelscited by 36
  20. 2024Digger: Detecting Copyright Content Mis-usage in Large Language Model Trainingcited by 24
  21. 2024D'ej`a Vu Memorization in Vision-Language Modelscited by 8

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025RECAP: Reproducing Copyrighted Data from LLMs Training with an Agentic Pipelinecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Erase Persona, Forget Lore: Benchmarking Multimodal Copyright Unlearning in Large Vision Language Models
  2. 2025Bridging the Copyright Gap: Do Large Vision-Language Models Recognize and Respect Copyrighted Content?
  3. 2025Tracking the Copyright of Large Vision-Language Models through Parameter Learning Adversarial Images
  4. 2025Trust the Model: Compact VLMs as In-Context Judges for Image-Text Data Quality
  5. 2025VLMs-in-the-Wild: Bridging the Gap Between Academic Benchmarks and Enterprise Reality
  6. 202620/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone
  7. 2025Contamination Detection for VLMs using Multi-Modal Semantic Perturbation
  8. 2026When VLMs Trust Context: Evaluating Scene Text Recognition under Misleading Context