mlsanityData to 7 Oct 2026

Can Language Models be Instructed to Protect Personal Information?

arXiv 2310.02224 ↗First posted 3 Oct 2023cs.CLcited by 46

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Large multimodal language models have proven transformative in numerous applications. However, these models have been shown to memorize and leak pre-training data, raising serious user privacy and information security concerns. While data leaks should be prevented, it is also crucial to examine the trade-off between the privacy protection and model utility of proposed approaches. In this paper, we introduce PrivQA -- a multimodal benchmark to assess this privacy/utility trade-off when a model is instructed to protect specific categories of personal information in a simulated scenario. We also propose a technique to iteratively self-moderate responses, which significantly improves privacy. However, through a series of red-teaming experiments, we find that adversaries can also easily circumvent these protections with simple jailbreaking methods through textual and/or image inputs. We believe PrivQA has the potential to support the development of new models with improved privacy protections, as well as the adversarial robustness of these protections. We release the entire PrivQA dataset at https://llm-access-control.github.io/.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 50 of its 65 references that are in the map. First posted on arXiv 3 Oct 2023.

Builds on

50 of 65 references.

References · 50

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  11. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  12. 2016Deep Learning with Differential Privacycited by 2,682
38 more
  1. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  2. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  3. 2020Extracting Training Data from Large Language Modelscited by 2,144
  4. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  5. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  6. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  7. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  8. 2019Machine Unlearningcited by 1,020
  9. 2022Red Teaming Language Models with Language Modelscited by 983
  10. 2022Mass-Editing Memory in a Transformercited by 976
  11. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  12. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  13. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  14. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  15. 2021Editing Factual Knowledge in Language Modelscited by 548
  16. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  17. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  18. 2022Memory-Based Model Editing at Scalecited by 443
  19. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 424
  20. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  21. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  22. 2023OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documentscited by 316
  23. 2020Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrievalcited by 305
  24. 2020Modifying Memories in Transformer Modelscited by 222
  25. 2020HateCheck: Functional Tests for Hate Speech Detection Modelscited by 215
  26. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 203
  27. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 194
  28. 2017Constructing Datasets for Multi-hop Reading Comprehension Across Documentscited by 190
  29. 2020VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrievalcited by 188
  30. 2020Recipes for Safety in Open-domain Chatbotscited by 168
  31. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  32. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  33. 2023Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entitiescited by 112
  34. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 111
  35. 2023EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Modelscited by 97
  36. 2023KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignmentcited by 94
  37. 2023FLIRT: Feedback Loop In-context Red Teamingcited by 81
  38. 2023LLM Censorship: A Machine Learning Challenge or a Computer Security Problem?cited by 47

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 20; 2025: 16; 2026: 3.

46 papers.

Sort:

By area:

  1. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 255
  2. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 218
  3. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  4. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 114
  5. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Spacecited by 88
  6. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  7. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  8. 2024Safety of Multimodal Large Language Models on Images and Textscited by 70
  9. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  10. 2024Red Teaming Visual Language Modelscited by 62
  11. 2024MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Modelscited by 51
  12. 2023Reducing Privacy Risks in Online Self-Disclosures with Language Modelscited by 47
34 more
  1. 2023Benefits and Harms of Large Language Models in Digital Mental Healthcited by 39
  2. 2023Exploiting Novel GPT-4 APIscited by 27
  3. 2024Security of AI Agentscited by 24
  4. 2024Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Modelscited by 23
  5. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 22
  6. 2024The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defensecited by 20
  7. 2024Granular Privacy Control for Geolocation with Vision Language Modelscited by 19
  8. 2025Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearningcited by 16
  9. 2024A Survey on Safe Multi-Modal Learning Systemcited by 15
  10. 2024Rag and Roll: An End-to-End Evaluation of Indirect Prompt Manipulations in LLM-based Application Frameworkscited by 14
  11. 2023NERIF: GPT-4V for Automatic Scoring of Drawn Modelscited by 13
  12. 2025Exploring User Security and Privacy Attitudes and Concerns Toward the Use of General-Purpose LLM Chatbots for Mental Healthcited by 12
  13. 2025Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluationcited by 11
  14. 2024Prompt Obfuscation for Large Language Modelscited by 10
  15. 2024Human-Centered Privacy Research in the Age of Large Language Modelscited by 9
  16. 2025MSTS: A Multimodal Safety Test Suite for Vision-Language Modelscited by 9
  17. 2024A Review of the Challenges with Massive Web-mined Corpora Used in Large Language Models Pre-Trainingcited by 7
  18. 2025Permissioned LLMs: Enforcing Access Control in Large Language Modelscited by 7
  19. 2025Safe-LLaVA: A Privacy-Preserving Vision-Language Dataset and Benchmark for Biometric Safetycited by 7
  20. 2025REVAL: A Comprehension Evaluation on Reliability and Values of Large Vision-Language Modelscited by 5
  21. 2025Beyond Text: Unveiling Privacy Vulnerabilities in Multi-modal Retrieval-Augmented Generationcited by 5
  22. 2025Language Models can Self-Improve at State-Value Estimation for Better Searchcited by 4
  23. 2025Unveiling Trust in Multimodal Large Language Models: Evaluation, Analysis, and Mitigationcited by 4
  24. 2025SafeMT: Multi-turn Safety for Multimodal Language Modelscited by 4
  25. 2025Watch Out Your Album! On the Inadvertent Privacy Memorization in Multi-Modal Large Language Modelscited by 3
  26. 2023MetaAID 2.5: A Secure Framework for Developing Metaverse Applications via Large Language Modelscited by 2
  27. 2024DOMBA: Double Model Balancing for Access-Controlled Language Models via Minimum-Bounded Aggregationcited by 2
  28. 2024Knowledge Graph Guided Evaluation of Abstention Techniquescited by 2
  29. 2025Role-Aware Language Models for Secure and Contextualized Access Control in Organizationscited by 2
  30. 2025A Survey on Training-free Alignment of Large Language Modelscited by 2
  31. 2025Measuring Physical-World Privacy Awareness of Large Language Models: An Evaluation Benchmarkcited by 2
  32. 2026Fix Before Search: Benchmarking Agentic Query Visual Pre-processing in Multimodal Retrieval-augmented Generationcited by 1
  33. 2026Safety-Potential Pruning for Enhancing Safety Prompts Against VLM Jailbreaking Without Retrainingcited by 0
  34. 2026CounterMoral: Editing Morals in Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Privacy-Aware Visual Language Models
  2. 2026Unveiling Privacy Risks in Multi-modal Large Language Models: Task-specific Vulnerabilities and Mitigation Challenges
  3. 2025Defeating Cerberus: Concept-Guided Privacy-Leakage Mitigation in Multimodal Language Models
  4. 2025Dual-Priv Pruning : Efficient Differential Private Fine-Tuning in Multimodal Large Language Models
  5. 2025Assessing Visual Privacy Risks in Multimodal AI: A Novel Taxonomy-Grounded Evaluation of Vision-Language Models
  6. 2026Privacy Collapse: Benign Fine-Tuning Can Break Contextual Privacy in Language Models
  7. 2026Seeing Without Exposing: Adaptive Privacy Control for Open-World, Context-Hungry MLLMs
  8. 2024Multi-PA: A Multi-perspective Benchmark on Privacy Assessment for Large Vision-Language Models