mlsanityData to 14 Oct 2026

SUA: Stealthy Multimodal Large Language Model Unlearning Attack

arXiv 2506.17265 ↗First posted 10 Jun 2025cs.LG cs.AIcited by 4

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) trained on massive data may memorize sensitive personal information and photos, posing serious privacy risks. To mitigate this, MLLM unlearning methods are proposed, which fine-tune MLLMs to reduce the ``forget'' sensitive information. However, it remains unclear whether the knowledge has been truly forgotten or just hidden in the model. Therefore, we propose to study a novel problem of LLM unlearning attack, which aims to recover the unlearned knowledge of an unlearned LLM. To achieve the goal, we propose a novel framework Stealthy Unlearning Attack (SUA) framework that learns a universal noise pattern. When applied to input images, this noise can trigger the model to reveal unlearned content. While pixel-level perturbations may be visually subtle, they can be detected in the semantic embedding space, making such attacks vulnerable to potential defenses. To improve stealthiness, we introduce an embedding alignment loss that minimizes the difference between the perturbed and denoised image embeddings, ensuring the attack is semantically unnoticeable. Experimental results show that SUA can effectively recover unlearned information from MLLMs. Furthermore, the learned noise generalizes well: a single perturbation trained on a subset of samples can reveal forgotten content in unseen images. This indicates that knowledge reappearance is not an occasional failure, but a consistent behavior.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 41 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

29 of 41 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models16 Feb
  2. 2024Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench29 Oct
  3. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov

Other references · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  6. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,358
  7. 2024TOFU: A Task of Fictitious Unlearning for LLMscited by 533
  8. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 504
  9. 2017Defense against Adversarial Attacks Using High-Level Representation Guided Denoisercited by 329
  10. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  11. 2024An Adversarial Perspective on Machine Unlearning for AI Safetycited by 123
  12. 2024Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)cited by 119
14 more
  1. 2015An Optimal Algorithm for Bandit and Zero-Order Convex Optimization with Two-Point Feedbackcited by 111
  2. 2024On Protecting the Data Privacy of Large Language Models (LLMs): A Surveycited by 96
  3. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Spacecited by 91
  4. 2024Catastrophic Failure of LLM Unlearning via Quantizationcited by 81
  5. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 76
  6. 2024DE-COP: Detecting Copyrighted Content in Language Models Training Datacited by 74
  7. 2024Demystifying Verbatim Memorization in Large Language Modelscited by 56
  8. 2024UnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AIcited by 51
  9. 2024CLEAR: Character Unlearning in Textual and Visual Modalitiescited by 39
  10. 2024Does Unlearning Truly Unlearn? A Black Box Evaluation of LLM Unlearning Methodscited by 26
  11. 2025LanP: Rethinking the Impact of Language Priors in Large Vision-Language Modelscited by 14
  12. 2025A General Framework to Enhance Fine-tuning-based LLM Unlearningcited by 14
  13. 2025Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedycited by 9
  14. 2025How Far are LLMs from Real Search? A Comprehensive Study on Efficiency, Completeness, and Inherent Capabilitiescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarkscited by 7
  2. 2026Your "Pro" LLM Subscription May Actually Be "Free": Exposing Fingerprint Spoofing Risks in LLM Inference Servicescited by 3
  3. 2026Copyright Detective: A Forensic System to Evidence LLMs Flickering Copyright Leakage Riskscited by 1
  4. 2026Unlearning in LLMs: Methods, Evaluation, and Open Challengescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning
  2. 2026Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearning
  3. 2026Exploring and Bridging Knowledge Holes in Unlearned Multimodal Large Language Models
  4. 2025Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation
  5. 2026Robust LLM Unlearning Against Relearning Attacks: The Minor Components in Representations Matter
  6. 2026POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shaking
  7. 2024Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models
  8. 2025Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning