mlsanityData to 7 Oct 2026

When Recovery Matters: The Blind Spot of Surrogate Privacy in MLLM Editing

arXiv 2606.07171 ↗First posted 5 Jun 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) enable flexible instruction-driven image editing, but privacy risks arise when user images expose diverse and user-specific private content. Canonical privacy protection strategies typically substitute sensitive regions with surrogate content before cloud editing. Yet, the resulting output is often an edited surrogate rather than the desired edited source image, neglecting the local recovery in both design and evaluation scope. To this end, we introduce SPPE (Surrogate-based Privacy-Preserving Editing), the first recovery-oriented benchmark covering 36 fine-grained privacy categories and 65 editing instructions. It defines two complementary tasks: 1) editability assessment, which estimates before cloud interaction whether a surrogate can induce an edit consistent with the original image; and 2) surrogate-to-source edit recovery, which evaluates whether the edited surrogate can be transferred back to the private source with the edit effect preserved. We address each task with a dedicated method: ERMA predicts surrogate editability through instruction-aware multimodal relation modeling, while method performs cycle-consistent recovery by using the surrogate editing pair as visual edit evidence and the source image as a source-preserving anchor. Experiments on SPPE and InstructPix2Pix show consistent improvements on both tasks. For editability assessment, ERMA improves over the best-performing baselines by 13.9% in SRCC and 12.3% in PLCC. For surrogate-to-source edit recovery, C2E-S2SER outperforms SOER across all 8 source integrity and edit consistency metrics on SPPE.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 46 of its 63 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

46 of 63 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Dual-Priv Pruning : Efficient Differential Private Fine-Tuning in Multimodal Large Language Models8 Jun
  2. 2024FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models3 Oct

Other references · 44

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  4. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  5. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  6. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  7. 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 792
  8. 2022MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessmentcited by 514
  9. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  10. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  11. 2017NIMA: Neural Image Assessmentcited by 387
  12. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 321
32 more
  1. 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 317
  2. 2023Generating Images with Multimodal Language Modelscited by 259
  3. 2022Visual Prompting via Image Inpaintingcited by 248
  4. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
  5. 2023TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessmentcited by 220
  6. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  7. 2023Making LLaMA SEE and Draw with SEED Tokenizercited by 184
  8. 2023SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Modelscited by 174
  9. 2023What Makes Good Examples for Visual In-Context Learning?cited by 150
  10. 2023Planting a SEED of Vision in Large Language Modelcited by 140
  11. 2023Cross-Image Attention for Zero-Shot Appearance Transfercited by 135
  12. 2023In-Context Learning Unlocked for Diffusion Modelscited by 103
  13. 2017Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Imagescited by 101
  14. 2024Zero-shot Image Editing with Reference Imitationcited by 64
  15. 2022Attentions Help CNNs See Better: Attention-based Hybrid Image Quality Assessment Networkcited by 57
  16. 2019Live Face De-Identification in Videocited by 48
  17. 2022Shift-tolerant Perceptual Similarity Metriccited by 44
  18. 2020Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Networkcited by 36
  19. 2024FreeEdit: Mask-free Reference-based Image Editing with Multi-modal Instructioncited by 35
  20. 2025Toward Generalized Image Quality Assessment: Relaxing the Perfect Reference Quality Assumptioncited by 34
  21. 2017Connecting Pixels to Privacy and Utility: Automatic Redaction of Private Information in Imagescited by 24
  22. 2025X2Edit: Revisiting Arbitrary-Instruction Image Editing through Self-Constructed Data and Task-Aware Representation Learningcited by 24
  23. 2025Edit Transfer: Learning Image Editing via Vision In-Context Relationscited by 23
  24. 2022Realistic Full-Body Anonymization with Surface-Guided GANscited by 15
  25. 2024Unleashing In-context Learning of Autoregressive Models for Few-shot Image Manipulationcited by 13
  26. 2023Differentially Private Decoupled Graph Convolutions for Multigranular Topology Protectioncited by 12
  27. 2025ReVision: A Dataset and Baseline VLM for Privacy-Preserving Task-Oriented Visual Instruction Rewritingcited by 6
  28. 2024BIV-Priv-Seg: Locating Private Content in Images Taken by People With Visual Impairmentscited by 5
  29. 2025PIXELS: Progressive Image Xemplar-based Editing with Latent Surgerycited by 5
  30. 2025Evaluation of Human Visual Privacy Protection: A Three-Dimensional Framework and Benchmark Datasetcited by 5
  31. 2024SpecRef: A Fast Training-free Baseline of Specific Reference-Condition Real Image Editingcited by 4
  32. 2025When Privacy Meets Recovery: The Overlooked Half of Surrogate-Driven Privacy Preservation for MLLM Editingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Unsafe2Safe: Controllable Image Anonymization for Downstream Utility
  2. 2026NullEdit: Stealthy Image Protection via VLM Condition Redirection
  3. 2026Seeing Without Exposing: Adaptive Privacy Control for Open-World, Context-Hungry MLLMs
  4. 2026Purify Once, Edit Freely: Breaking Image Protections under Model Mismatch
  5. 2026When Visual Privacy Protection Meets Multimodal Large Language Models
  6. 2025On the Robustness of Vision-Language Models in Zero-shot Privacy Classification
  7. 2023EditShield: Protecting Unauthorized Image Editing by Instruction-guided Diffusion Models
  8. 2024Multi-PA: A Multi-perspective Benchmark on Privacy Assessment for Large Vision-Language Models