mlsanityData to 14 Oct 2026

Is Our Benchmark Enough? An Analysis of Continual Learning for MLLMs

arXiv 2606.20961 ↗First posted 18 Jun 2026cs.LG cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Continual adaptation is essential for multimodal large language models (MLLMs) deployed across evolving domains, but the state-of-the-art MR-LoRA method highly relies on the assumption that a MLLM-based router is necessary to process complex multimodal inputs. This paper revisits this claim on the MLLM-CL benchmark and argues for two claims. First, routing does not require an MLLM: a simple training-free, replay-free ptotypical routing method (textscRePRo), uses frozen pretrained features and task prototypes to match the MLLM-based router of MR-LoRA at far lower computational cost. Second, shared experts do not improve continual learning for MLLMs, despite their theoretical appeal. We show that these findings arise from two structural limitations of MLLM-CL: (1) its tasks are highly separable in representation space, and (2) its fixed task order makes conclusions sensitive to a single curriculum rather than robust across diverse continual-learning trajectories. As a result, the benchmark primarily rewards learning in isolation rather than genuine continual transfer. This motivates a new design for future benchmarks of continual MLLM learning, with overlapping task manifolds, multiple task orders, fine-grained domain shifts, and evaluation protocols that reward forward transfer as well as retention.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 25 of its 36 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

25 of 36 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025MLLM-CL: Continual Learning for Multimodal Large Language Models5 Jun
  2. 2025SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning5 May
  3. 2024CoIN: A Benchmark of Continual Instruction tuNing for Multimodel Large Language Model13 Mar

Other references · 22

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  6. 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 312
  7. 2024InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learningcited by 198
  8. 2024Mixture of LoRA Expertscited by 181
  9. 2024MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Expertscited by 147
  10. 2023When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applicationscited by 111
  11. 2024Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Expertscited by 108
  12. 2023Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuningcited by 106
10 more
  1. 2022CLiMB: A Continual Learning Benchmark for Vision-and-Language Taskscited by 68
  2. 2025SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learningcited by 63
  3. 2024Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Modelscited by 38
  4. 2024AdaMoLE: Fine-Tuning Large Language Models with Adaptive Mixture of Low-Rank Adaptation Expertscited by 34
  5. 2023SiRA: Sparse Mixture of Low Rank Adaptationcited by 32
  6. 2024Online-LoRA: Task-free Online Continual Learning via Low Rank Adaptationcited by 27
  7. 2025LoRA Subtraction for Drift-Resistant Space in Exemplar-Free Continual Learningcited by 18
  8. 2025MCITlib: Multimodal Continual Instruction Tuning Library and Benchmarkcited by 13
  9. 2024Learning Attentional Mixture of LoRAs for Language Model Continual Learningcited by 10
  10. 2025MoA: Heterogeneous Mixture of Adapters for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025When Continue Learning Meets Multimodal Large Language Model: A Survey
  2. 2025Routing-Based Continual Learning for Multimodal Large Language Models
  3. 2025Merge then Realign: Simple and Effective Modality-Incremental Continual Learning for Multimodal LLMs
  4. 2026Continual-NExT: A Unified Comprehension And Generation Continual Learning Framework
  5. 2024Improving Multimodal Large Language Models Using Continual Learning
  6. 2024Modality-Inconsistent Continual Learning of Multimodal Large Language Models
  7. 2026Hidden Forgetting in Continual Multimodal Learning: When Accuracy Survives but Grounding Fails
  8. 2025Continual Learning for VLMs: A Survey and Taxonomy Beyond Forgetting