Exploring and Bridging Knowledge Holes in Unlearned Multimodal Large Language Models
On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness
also in 1 more
Abstract (arXiv)
Machine unlearning offers a promising approach to remove unsafe content from Multimodal Large Language Models (MLLMs), yet ensuring the precision of unlearning remains a persistent challenge. One reason is that current MLLM unlearning evaluation paradigms suffer from a critical blind spot: they assess model utility through benchmarks whose representations are distant from the forget set, failing to capture knowledge holes---severe degradation on benign adjacent inputs. To probe knowledge holes in unlearned MLLMs, we construct a benchmark that captures unintended degradation on benign inputs sharing generic patterns with the forget set, and confirm through controlled experiments that they are a systematic consequence of commonly used approaches. Furthermore, to bridge this gap, we propose Selective Protection with Anchored Regularization, which protects generic patterns via anchored activation filtering while reinforcing them through entity-abstracted enhancement. Our experiments on SafeEraser demonstrate that SPAR recovers over 98% of vanilla response quality compared to below 50% for standard baselines---while achieving 0.00% attack success rate and competitive model utility. These results underscore the necessity of more fine-grained evaluation for trustworthy MLLM unlearning.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 25 references that are in the map. First posted on arXiv 3 Aug 2026.
Builds on
14 of 25 references.
Closest ancestors · 5
Same direction, newest first
- 2026Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearning26 May
- 2025Hierarchy-Aware Multimodal Unlearning for Medical AI10 Dec
- 2025From Learning to Unlearning: Biomedical Security Protection in Multimodal Large Language Models6 Aug
- 2024Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset5 Nov
- 2024Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench29 Oct
Other references · 9
- 2019Machine Unlearningcited by 1,020
- 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
- 2024TOFU: A Task of Fictitious Unlearning for LLMscited by 507
- 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 484
- 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
- 2023Who's Harry Potter? Approximate Unlearning in LLMscited by 366
- 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lenscited by 139
- 2025Probing Knowledge Holes in Unlearned LLMscited by 4
- 2026Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearningcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning
- 2025Towards Reasoning-Preserving Unlearning in Multimodal Large Language Models
- 2026Open-MMUnlearning: Unifying Methods and Evaluation for MLLM Unlearning
- 2025MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering
- 2025MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models
- 2026MLUBench: A Benchmark for Lifelong Unlearning Evaluation in MLLMs
- 2024RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models
- 2025Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning