mlsanityData to 7 Oct 2026

Extracting Training Data from Document-Based VQA Models

arXiv 2407.08707 ↗First posted 11 Jul 2024cs.CV cs.LGcited by 6

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Vision-Language Models (VLMs) have made remarkable progress in document-based Visual Question Answering (i.e., responding to queries about the contents of an input document provided as an image). In this work, we show these models can memorize responses for training samples and regurgitate them even when the relevant visual information has been removed. This includes Personal Identifiable Information (PII) repeated once in the training set, indicating these models could divulge memorised sensitive information and therefore pose a privacy risk. We quantitatively measure the extractability of information in controlled experiments and differentiate between cases where it arises from generalization capabilities or from memorization. We further investigate the factors that influence memorization across multiple state-of-the-art models and propose an effective heuristic countermeasure that empirically prevents the extractability of PII.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 37 references that are in the map. First posted on arXiv 11 Jul 2024.

Builds on

32 of 37 references.

References · 32

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  3. 2020Extracting Training Data from Large Language Modelscited by 2,144
  4. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  5. 2023PaLM 2 Technical Reportcited by 1,154
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  7. 2021Membership Inference Attacks From First Principlescited by 839
  8. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  9. 2023Extracting Training Data from Diffusion Modelscited by 723
  10. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  11. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 481
  12. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
20 more
  1. 2021InfographicVQAcited by 430
  2. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  3. 2021OCR-free Document Understanding Transformercited by 400
  4. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 360
  5. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  6. 2020Counterfactual VQA: A Cause-Effect Look at Language Biascited by 266
  7. 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 254
  8. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
  9. 2023Understanding and Mitigating Copying in Diffusion Modelscited by 205
  10. 2021Counterfactual Memorization in Neural Language Modelscited by 155
  11. 2020Counterfactual Samples Synthesizing for Robust Visual Question Answeringcited by 153
  12. 2022Hierarchical multimodal transformers for Multi-Page DocVQAcited by 102
  13. 2022End-to-end Document Recognition and Understanding with Dessurtcited by 73
  14. 2021Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answeringcited by 71
  15. 2020OCR Post Correction for Endangered Language Textscited by 31
  16. 2022M^4I: Multi-modal Models Membership Inferencecited by 30
  17. 2022Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQAcited by 30
  18. 2023Do SSL Models Have D'ej`a Vu? A Case of Unintended Memorization in Self-supervised Learningcited by 19
  19. 2023Privacy-Aware Document Visual Question Answeringcited by 17
  20. 2022You Are What You Write: Preserving Privacy in the Era of Large Language Modelscited by 11

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 4.

6 papers.

  1. 2024Copyright-Protected Language Generation via Adaptive Model Fusioncited by 6
  2. 2025Membership Inference Attacks on Sequence Modelscited by 6
  3. 2025Watch Out Your Album! On the Inadvertent Privacy Memorization in Multi-Modal Large Language Modelscited by 3
  4. 2024NeurIPS 2023 Competition: Privacy Preserving Federated Learning Document VQAcited by 2
  5. 2025A Lightweight Method to Disrupt Memorized Sequences in LLMcited by 1
  6. 2025DIS-CO: Discovering Copyrighted Content in VLMs Training Datacited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Undesirable Memorization in Large Language Models: A Survey
  2. 2024D'ej`a Vu Memorization in Vision-Language Models
  3. 2025DocMIA: Document-Level Membership Inference Attacks against DocVQA Models
  4. 2024The Phantom Menace: Unmasking Privacy Leakages in Vision-Language Models
  5. 2026Beyond Visual Evidence: Revealing and Mitigating Relational Privacy Leakage in Document MLLMs
  6. 2026Doc-PP: Document Policy Preservation Benchmark for Large Vision-Language Models
  7. 2025Quantifying Memorization and Parametric Response Rates in Retrieval-Augmented Vision-Language Models
  8. 2025Joint Extraction Matters: Prompt-Based Visual Question Answering for Multi-Field Document Information Extraction