mlsanityData to 7 Oct 2026

POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shaking

arXiv 2607.06649 ↗First posted 7 Jul 2026cs.CR cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have demonstrated impressive performance on cross-modal tasks by jointly training on large-scale textual and visual data, where privacy-sensitive examples could be unintentionally encoded, raising concerns about privacy or copyright violation. To this end, Multi-modality Machine Unlearning (MMU) was proposed as a mitigation that can effectively force MLLMs to forget private information. However, the robustness of such unlearning methods is not fully exploited when the model is published and accessible to malicious users. In this paper, we propose a novel adversarial strategy, namely Prompt-Optimized Parameter Shaking (POPS), aiming to recover the supposedly unlearned multi-modality knowledge from the MLLMs. Our method elicits the victim MLLMs to generate potential private examples via prompt-suffix optimization, and then exploits these synthesized outputs to fine-tune the models so they disclose the true private information. The experiments on the different MMU benchmarks reveal substantial weaknesses in the existing MMU algorithms. Our POPS can even achieve a near-complete recovery of supposedly erased sensitive information on the unlearned MLLMs, exposing fundamental vulnerabilities that challenge the foundational robustness of representative MMU-based privacy protections.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 41 of its 49 references that are in the map. First posted on arXiv 7 Jul 2026.

Builds on

41 of 49 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation1 May
  2. 2025Survey of Adversarial Robustness in Multimodal Large Language Models18 Mar
  3. 2025Modality-Aware Neuron Pruning for Unlearning in Multimodal Large Language Models21 Feb
  4. 2025MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models16 Feb
  5. 2024Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench29 Oct
  6. 2024Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models21 May

Other references · 35

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2024GPT-4o System Cardcited by 4,467
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  5. 2020Extracting Training Data from Large Language Modelscited by 2,144
  6. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  7. 2019Machine Unlearningcited by 1,020
  8. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  9. 2023Extracting Training Data from Diffusion Modelscited by 723
  10. 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
  11. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 484
  12. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generationcited by 376
23 more
  1. 2022A Survey of Machine Unlearningcited by 298
  2. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearningcited by 274
  3. 2024AnyGPT: Unified Multimodal LLM with Discrete Sequence Modelingcited by 213
  4. 2021MultiModalQA: Complex Question Answering over Text, Tables and Imagescited by 177
  5. 2022Continual Learning and Private Unlearningcited by 124
  6. 2024Machine Unlearning of Pre-trained Large Language Modelscited by 118
  7. 2020Variational Bayesian Unlearningcited by 114
  8. 2024Soft Prompting for Unlearning in Large Language Modelscited by 42
  9. 2024CLEAR: Character Unlearning in Textual and Visual Modalitiescited by 36
  10. 2020Situated and Interactive Multimodal Conversationscited by 35
  11. 2024MLLM-CompBench: A Comparative Reasoning Benchmark for Multimodal LLMscited by 32
  12. 2025The European Union general data protection regulation: what it is and what it meanscited by 30
  13. 2024Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Modelscited by 24
  14. 2023MultiDelete for Multimodal Machine Unlearningcited by 20
  15. 2024Shake to Leak: Fine-tuning Diffusion Models Can Amplify the Generative Privacy Riskcited by 17
  16. 2022Multimodal Conversational AI: A Survey of Datasets and Approachescited by 16
  17. 2023X4D-SceneFormer: Enhanced Scene Understanding on 4D Point Cloud Videos through Cross-modal Knowledge Transfercited by 12
  18. 2024Model Inversion Attacks: A Survey of Approaches and Countermeasurescited by 11
  19. 2024CrossVideo: Self-supervised Cross-modal Contrastive Learning for Point Cloud Video Understandingcited by 6
  20. 2024State-of-the-Art Approaches to Enhancing Privacy Preservation of Machine Learning Datasets: A Surveycited by 5
  21. 2024Multi-Modal Recommendation Unlearning for Legal, Licensing, and Modality Constraintscited by 5
  22. 2024MMCert: Provable Defense against Adversarial Attacks to Multi-modal Modelscited by 4
  23. 2024Multi-modal Semantic Understanding with Contrastive Cross-modal Feature Alignmentcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Open-MMUnlearning: Unifying Methods and Evaluation for MLLM Unlearning
  2. 2025SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning
  3. 2025SUA: Stealthy Multimodal Large Language Model Unlearning Attack
  4. 2026Exploring and Bridging Knowledge Holes in Unlearned Multimodal Large Language Models
  5. 2026Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearning
  6. 2025Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning
  7. 2026PPE-Bench: A Benchmark for Evaluating MLLM Unlearning under Private-Public Entanglement
  8. 2025PEBench: A Fictitious Dataset to Benchmark Machine Unlearning for Multimodal Large Language Models