# Exploring and Bridging Knowledge Holes in Unlearned Multimodal Large Language Models

arXiv: https://arxiv.org/abs/2608.01849 · HTML: https://mlsanity.com/paper/2608.01849/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 3 Aug 2026.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)
- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 14 of 14)

- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [VizWiz Grand Challenge: Answering Visual Questions from Blind People](https://mlsanity.com/paper/1802.08218/) (2018)
- [TOFU: A Task of Fictitious Unlearning for LLMs](https://mlsanity.com/paper/2401.06121/) (2024)
- [Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning](https://mlsanity.com/paper/2404.05868/) (2024)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Who's Harry Potter? Approximate Unlearning in LLMs](https://mlsanity.com/paper/2310.02238/) (2023)
- [Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens](https://mlsanity.com/paper/2411.16724/) (2024)
- [Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench](https://mlsanity.com/paper/2410.22108/) (2024)
- [Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset](https://mlsanity.com/paper/2411.03554/) (2024)
- [Probing Knowledge Holes in Unlearned LLMs](https://mlsanity.com/paper/2511.00030/) (2025)
- [From Learning to Unlearning: Biomedical Security Protection in Multimodal Large Language Models](https://mlsanity.com/paper/2508.04192/) (2025)
- [Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearning](https://mlsanity.com/paper/2605.05909/) (2026)
- [Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearning](https://mlsanity.com/paper/2606.00105/) (2026)
- [Hierarchy-Aware Multimodal Unlearning for Medical AI](https://mlsanity.com/paper/2512.09867/) (2025)

## Abstract (arXiv)

Machine unlearning offers a promising approach to remove unsafe content from Multimodal Large Language Models (MLLMs), yet ensuring the precision of unlearning remains a persistent challenge. One reason is that current MLLM unlearning evaluation paradigms suffer from a critical blind spot: they assess model utility through benchmarks whose representations are distant from the forget set, failing to capture knowledge holes---severe degradation on benign adjacent inputs. To probe knowledge holes in unlearned MLLMs, we construct a benchmark that captures unintended degradation on benign inputs sharing generic patterns with the forget set, and confirm through controlled experiments that they are a systematic consequence of commonly used approaches. Furthermore, to bridge this gap, we propose Selective Protection with Anchored Regularization, which protects generic patterns via anchored activation filtering while reinforcing them through entity-abstracted enhancement. Our experiments on SafeEraser demonstrate that SPAR recovers over 98% of vanilla response quality compared to below 50% for standard baselines---while achieving 0.00% attack success rate and competitive model utility. These results underscore the necessity of more fine-grained evaluation for trustworthy MLLM unlearning.