mlsanityData to 7 Oct 2026

Imperceptible and Reversible Adversarial Examples against Vision-Language Models for Privacy Protection

arXiv 2607.10329 ↗First posted 11 Jul 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Vision Language Models (VLMs) offer powerful multimodal ability but also expose users to text-based privacy attacks where adversaries crawl online photos and query VLMs to extract sensitive attributes. Existing reversible adversarial example (RAE) methods protect images in purely visual tasks but fail in multimodal settings, and current adversarial examples on VLMs rely on high frequency noise that severely degrades visual quality. We propose CloakDiff, the first framework for reversible, high fidelity privacy protection against text-based query attacks in VLMs. CloakDiff produces imperceptible adversarial examples by combining diffusion based adversarial editing with an invertible network that embeds the original image for lossless recovery. It perturbs both pixel space embeddings and manipulates latent cross attention maps to ensure strong cross-model and cross-prompt transferability while preserving global visual structure. To further enhance fidelity, we design EDM Heuristic Sampling, a principled diffusion schedule for adversarial guidance. Experiments on multiple datasets and VLMs demonstrate that CloakDiff delivers multimodal privacy preservation with high visual quality and reversibility.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 63 references that are in the map. First posted on arXiv 11 Jul 2026.

Builds on

14 of 63 references.

References · 14

  1. 2020Denoising Diffusion Implicit Modelscited by 9,655
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2022Classifier-Free Diffusion Guidancecited by 5,717
  4. 2025Qwen2.5-VL Technical Reportcited by 5,418
  5. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  6. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  7. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  8. 2018Analyzing Inverse Problems with Invertible Neural Networkscited by 239
  9. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 167
  10. 2016Technical Report on the CleverHans v2.1.0 Adversarial Examples Librarycited by 123
  11. 2022Progressive Distillation for Fast Sampling of Diffusion Modelscited by 61
  12. 2025Spa-VLM: Stealthy Poisoning Attacks on RAG-based VLMcited by 10
2 more
  1. 2026Transferable Physical-World Adversarial Patches Against Pedestrian Detection Modelscited by 2
  2. 2026Transferable Physical-World Adversarial Patches Against Object Detection in Autonomous Drivingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025VIP: Visual Information Protection through Adversarial Attacks on Vision-Language Models
  2. 2023DiffProtect: Generate Adversarial Examples with Diffusion Models for Facial Privacy Protection
  3. 2024Privacy Protection in Personalized Diffusion Models via Targeted Cross-Attention Adversarial Attack
  4. 2025Who Can See Through You? Adversarial Shielding Against VLM-Based Attribute Inference Attacks
  5. 2023Towards Prompt-robust Face Privacy Protection via Adversarial Decoupling Augmentation Framework
  6. 2025Attention! Your Vision Language Model Could Be Maliciously Manipulated
  7. 2024DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing
  8. 2024PID: Prompt-Independent Data Protection Against Latent Diffusion Models