# MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models

arXiv: https://arxiv.org/abs/2502.11051 · HTML: https://mlsanity.com/paper/2502.11051/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 42 of its 60 references that are in the map.
- First posted on arXiv 16 Feb 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)
- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 30 of 42)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [DeepSeek-VL: Towards Real-World Vision-Language Understanding](https://mlsanity.com/paper/2403.05525/) (2024)
- [TOFU: A Task of Fictitious Unlearning for LLMs](https://mlsanity.com/paper/2401.06121/) (2024)
- [The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning](https://mlsanity.com/paper/2403.03218/) (2024)
- [Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning](https://mlsanity.com/paper/2404.05868/) (2024)
- [SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation](https://mlsanity.com/paper/2310.12508/) (2023)
- [Who's Harry Potter? Approximate Unlearning in LLMs](https://mlsanity.com/paper/2310.02238/) (2023)
- [PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding](https://mlsanity.com/paper/2312.04461/) (2023)
- [Unrolling SGD: Understanding Factors Influencing Machine Unlearning](https://mlsanity.com/paper/2109.13398/) (2021)
- [In-Context Unlearning: Language Models as Few Shot Unlearners](https://mlsanity.com/paper/2310.07579/) (2023)
- [Towards Safer Large Language Models through Machine Unlearning](https://mlsanity.com/paper/2402.10058/) (2024)
- [Variational Bayesian Unlearning](https://mlsanity.com/paper/2010.12883/) (2020)
- [MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance](https://mlsanity.com/paper/2401.02906/) (2024)
- [KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment](https://mlsanity.com/paper/2305.06535/) (2023)
- [Guardrail Baselines for Unlearning in LLMs](https://mlsanity.com/paper/2403.03329/) (2024)
- [UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Web](https://mlsanity.com/paper/2310.18340/) (2023)
- [Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench](https://mlsanity.com/paper/2410.22108/) (2024)
- [A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges](https://mlsanity.com/paper/2412.11936/) (2024)
- [Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2412.02104/) (2024)
- [Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancement](https://mlsanity.com/paper/2409.19732/) (2024)
- [Digital Forgetting in Large Language Models: A Survey of Unlearning Methods](https://mlsanity.com/paper/2404.02062/) (2024)
- [Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Concepts](https://mlsanity.com/paper/2410.12777/) (2024)

## Built on by (most cited first, 30 of 30)

- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [LLM Agents for Education: Advances and Applications](https://mlsanity.com/paper/2503.11733/) (2025)
- [MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detection](https://mlsanity.com/paper/2503.18132/) (2025)
- [Are Multimodal Large Language Models Ready for Omnidirectional Spatial Reasoning?](https://mlsanity.com/paper/2505.11907/) (2025)
- [DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruning](https://mlsanity.com/paper/2509.23883/) (2025)
- [GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detection](https://mlsanity.com/paper/2505.13312/) (2025)
- [Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearning](https://mlsanity.com/paper/2503.11832/) (2025)
- [Towards Omnidirectional Reasoning with 360-R1: A Dataset, Benchmark, and GRPO-based Method](https://mlsanity.com/paper/2505.14197/) (2025)
- [HyperG: Hypergraph-Enhanced LLMs for Structured Knowledge](https://mlsanity.com/paper/2502.18125/) (2025)
- [MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering](https://mlsanity.com/paper/2510.04217/) (2025)
- [Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework](https://mlsanity.com/paper/2602.19549/) (2026)
- [OFFSIDE: Benchmarking Unlearning Misinformation in Multimodal Large Language Models](https://mlsanity.com/paper/2510.22535/) (2025)
- [CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoring](https://mlsanity.com/paper/2505.13965/) (2025)
- [SUA: Stealthy Multimodal Large Language Model Unlearning Attack](https://mlsanity.com/paper/2506.17265/) (2025)
- [Towards Reasoning-Preserving Unlearning in Multimodal Large Language Models](https://mlsanity.com/paper/2512.17911/) (2025)
- [Visual-Guided Key-Token Regularization for Multimodal Large Language Model Unlearning](https://mlsanity.com/paper/2601.22020/) (2026)
- [Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearning](https://mlsanity.com/paper/2605.05909/) (2026)
- [Position: The Term "Machine Unlearning" Is Overused in LLMs](https://mlsanity.com/paper/2606.27379/) (2026)
- [Mitigating Memorization Where It Happens](https://mlsanity.com/paper/2602.08901/) (2026)
- [Robust MLLM Unlearning via Visual Knowledge Distillation](https://mlsanity.com/paper/2512.11325/) (2025)
- [ICU-Bench:Benchmarking Continual Unlearning in Multimodal Large Language Models](https://mlsanity.com/paper/2605.05938/) (2026)
- [PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Models](https://mlsanity.com/paper/2605.08800/) (2026)
- [MLUBench: A Benchmark for Lifelong Unlearning Evaluation in MLLMs](https://mlsanity.com/paper/2606.12809/) (2026)
- [Hierarchical Dual-Strategy Unlearning for Biomedical and Healthcare Intelligence Using Imperfect and Privacy-Sensitive Medical Data](https://mlsanity.com/paper/2511.19498/) (2025)
- [SPACE: Source-free Proxy Anchor Concept Erasure for MLLMs](https://mlsanity.com/paper/2606.09868/) (2026)
- [POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shaking](https://mlsanity.com/paper/2607.06649/) (2026)
- [Stochastic Meta-Unlearning: Bridging Language Backbone and Multimodal Unlearning](https://mlsanity.com/paper/2607.18615/) (2026)
- [Partition-Aware Unlearning for Removing Spurious Correlations in Large Vision-Language Models](https://mlsanity.com/paper/2608.29996/) (2026)
- [SIEVE: Selective attention-value Suppression for Vision-Language Models Unlearning](https://mlsanity.com/paper/2610.01962/) (2026)
- [CUSE: Fine-Grained Multimodal Unlearning with Counterfactual Guidance and On-Policy Self-Distillation](https://mlsanity.com/paper/2610.14763/) (2026)

## Abstract (arXiv)

Recent progress in Machine Unlearning (MU) has introduced solutions for the selective removal of private or sensitive information encoded within deep neural networks. Nonetheless, MU for Multimodal Large Language Models (MLLMs) remains in its nascent phase. Therefore, we propose to reformulate the task of multimodal MU in the era of MLLMs, which aims to erase only the visual patterns associated with a given entity while preserving the corresponding textual knowledge encoded within the original parameters of the language model backbone. Furthermore, we develop a novel geometry-constrained gradient ascent method MMUnlearner. It updates the weights of MLLMs with a weight saliency map jointly restricted by the remaining concepts and textual knowledge during unlearning, thereby preserving parameters essential for non-target knowledge. Extensive experiments demonstrate that MMUnlearner surpasses baselines that finetuning MLLMs with VQA data directly through Gradient Ascent (GA) or Negative Preference Optimization (NPO), across all evaluation dimensions. Our code can be found in [this URL](https://github.com/Z1zs/MMUnlearner).