mlsanityData to 7 Oct 2026

MLUBench: A Benchmark for Lifelong Unlearning Evaluation in MLLMs

arXiv 2606.12809 ↗First posted 11 Jun 2026cs.AI cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Multimodal large language models (MLLMs) are trained on massive multimodal data, making data unlearning increasingly important as data owners may request the removal of specific content. In practice, these requests often arrive sequentially over time, giving rise to the challenging problem of MLLM Lifelong Unlearning. However, most existing benchmarks are limited in scale and scope, failing to capture the complexities of MLLM lifelong unlearning. To fill this gap, we introduce the MLUBench, a large-scale and comprehensive benchmark featuring 127 entities across 9 classes under lifelong unlearning requests. We perform extensive experiments using MLUBench and reveal that existing unlearning methods suffer from severe, cumulative degradation. More critically, we further identify the unique challenge of this problem: unlike in unimodal models, MLLM lifelong unlearning is constrained by the need to preserve multimodal alignment. Continually unlearning from one modality could degrade the entire model. To alleviate this challenge, we propose LUMoE, an effective method. Experiments demonstrate that LUMoE significantly mitigates the degradation problem faced by baselines. The source code and the MLUBench dataset are open-sourced in https://github.com/lihe-maxsize/Lifelong_Unlearning_main.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 44 of its 55 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

44 of 55 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025PULSE: Practical Evaluation Scenarios for Large Multimodal Model Unlearning2 Jul
  2. 2025MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models16 Feb
  3. 2024Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset5 Nov
  4. 2024Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench29 Oct
  5. 2024Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models21 May

Other references · 39

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2024GPT-4o System Cardcited by 4,467
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  7. 2025Qwen3-VL Technical Reportcited by 2,534
  8. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  9. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  10. 2023A Survey on Multimodal Large Language Modelscited by 898
  11. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  12. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
27 more
  1. 2024TOFU: A Task of Fictitious Unlearning for LLMscited by 507
  2. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 484
  3. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  4. 2023Large Language Model Unlearningcited by 288
  5. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 277
  6. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Modelscited by 251
  7. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  8. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
  9. 2022Fine-tuned Language Models are Continual Learnerscited by 152
  10. 2020A Neural Dirichlet Process Mixture Model for Task-Free Continual Learningcited by 133
  11. 2022Continual Learning and Private Unlearningcited by 124
  12. 2024Machine Unlearning of Pre-trained Large Language Modelscited by 118
  13. 2022TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Modelscited by 117
  14. 2021Lifelong Pretraining: Continually Adapting Language Models to Emerging Corporacited by 114
  15. 2023Lifelong Language Pretraining with Distribution-Specialized Expertscited by 64
  16. 2024CoIN: A Benchmark of Continual Instruction tuNing for Multimodel Large Language Modelcited by 60
  17. 2024Model Tailor: Mitigating Catastrophic Forgetting in Multi-modal Large Language Modelscited by 54
  18. 2024On Large Language Model Continual Unlearningcited by 51
  19. 2024Theory on Mixture-of-Experts in Continual Learningcited by 50
  20. 2024Machine Unlearning in Generative AI: A Surveycited by 49
  21. 2023TiC-CLIP: Continual Training of CLIP Modelscited by 38
  22. 2024Divide and not forget: Ensemble of selectively trained experts in Continual Learningcited by 38
  23. 2024Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspectivecited by 33
  24. 2023Can BERT Refrain from Forgetting on Sequential Tasks? A Probing Studycited by 11
  25. 2025Practical Continual Forgetting for Pre-trained Vision Modelscited by 4
  26. 2025Lifelong Learning with Task-Specific Adaptation: Addressing the Stability-Plasticity Dilemmacited by 4
  27. 2025Generative Model Unlearning: A Survey through Target Events, Unlearning Operators, and Evaluation Protocolscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Model Merging Approach for Continual MLLM Unlearning
  2. 2026Open-MMUnlearning: Unifying Methods and Evaluation for MLLM Unlearning
  3. 2026Exploring and Bridging Knowledge Holes in Unlearned Multimodal Large Language Models
  4. 2025PEBench: A Fictitious Dataset to Benchmark Machine Unlearning for Multimodal Large Language Models
  5. 2025Align-then-Unlearn: Embedding Alignment for LLM Unlearning
  6. 2024MileBench: Benchmarking MLLMs in Long Context
  7. 2025MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering
  8. 2026Unlearning Under Imbalance: Benchmarking Fairness in Multimodal LLM Unlearning