# Assessing Visual Privacy Risks in Multimodal AI: A Novel Taxonomy-Grounded Evaluation of Vision-Language Models

arXiv: https://arxiv.org/abs/2509.23827 · HTML: https://mlsanity.com/paper/2509.23827/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 7 of its 34 references that are in the map.
- First posted on arXiv 28 Sep 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)

## Builds on (most cited first, 7 of 7)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://mlsanity.com/paper/2204.07705/) (2022)
- [PaliGemma: A versatile 3B VLM for transfer](https://mlsanity.com/paper/2407.07726/) (2024)
- [Privacy-Aware Visual Language Models](https://mlsanity.com/paper/2405.17423/) (2024)

## Built on by (most cited first, 6 of 6)

- [MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Models](https://mlsanity.com/paper/2511.16940/) (2025)
- [Rethinking Visual Privacy: A Compositional Privacy Risk Framework for Severity Assessment with VLMs](https://mlsanity.com/paper/2603.21573/) (2026)
- [What Your Posts Reveal: A Benchmark and Agentic Framework for User-Level Privacy Leakage on Social Media](https://mlsanity.com/paper/2606.06784/) (2026)
- [On the Robustness of Vision-Language Models in Zero-shot Privacy Classification](https://mlsanity.com/paper/2510.09253/) (2025)
- [Position: Embodied AI Requires a Privacy-Utility Trade-off](https://mlsanity.com/paper/2605.05017/) (2026)
- [PrivLEX: Detecting legal concepts in images through Vision-Language Models](https://mlsanity.com/paper/2601.09449/) (2026)

## Abstract (arXiv)

Artificial Intelligence have profoundly transformed the technological landscape in recent years. Large Language Models (LLMs) have demonstrated impressive abilities in reasoning, text comprehension, contextual pattern recognition, and integrating language with visual understanding. While these advances offer significant benefits, they also reveal critical limitations in the models' ability to grasp the notion of privacy. There is hence substantial interest in determining if and how these models can understand and enforce privacy principles, particularly given the lack of supporting resources to test such a task. In this work, we address these challenges by examining how legal frameworks can inform the capabilities of these emerging technologies. To this end, we introduce a comprehensive, multi-level Visual Privacy Taxonomy that captures a wide range of privacy issues, designed to be scalable and adaptable to existing and future research needs. Furthermore, we evaluate the capabilities of several state-of-the-art Vision-Language Models (VLMs), revealing significant inconsistencies in their understanding of contextual privacy. Our work contributes both a foundational taxonomy for future research and a critical benchmark of current model limitations, demonstrating the urgent need for more robust, privacy-aware AI systems.