Beyond Routing Saturation: A Long-Horizon Class-Incremental Perspective on Expert Routing in Multimodal Continual Instruction Tuning
On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP
also in 1 more
Abstract (arXiv)
Multimodal Continual Instruction Tuning (MCIT) enables multimodal large language models to acquire new tasks sequentially while retaining previously learned capabilities. Many recent methods maintain task-specific LoRA experts and route each input to one or more experts at inference. Yet the task-identification problem underlying expert routing remains under-explored. We show that routing is nearly saturated on widely used MCIT benchmarks. Textual fingerprints that leak task identity and short 4--10-task sequences with few competing experts jointly obscure the long-horizon routing problem. To expose this challenge, we introduce FLEX (Fingerprint-reduced Long-horizon Expert eXamination), a 34-task long-horizon MCIT benchmark with weakened textual fingerprints. FLEX groups tasks with similar instruction and answer formats but diverse visual and knowledge domains, normalizes their outer templates, and evaluates routing over a substantially larger expert pool. Crucially, we formulate progressive-LoRA routing as soft task-as-class Multimodal Class-Incremental Learning (MCIL): each task defines an incremental routing class, whose complete score distribution supplies the LoRA mixture weights, with hard routing as a discrete special case. FLEX exposes this expanding task-identification challenge, while the MCIL formulation provides a principled interface for transferring CIL methods to expert routing. We instantiate PureLoRA as a controlled baseline and adapt four CIL methods to four MCIT frameworks without modifying their LoRA experts or generation pipelines. Our plug-in routers improve strict LoRA matching by up to 16.3 percentage points and overall MacroScore by up to 4.6 points. Code is available at: https://github.com/RINC-CL/FLEX
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 9 of its 114 references that are in the map. First posted on arXiv 2 Aug 2026.
Builds on
9 of 114 references.
Closest ancestors · 2
Same direction, newest first
- 2026Prism: A Plug-in Reproducible Infrastructure for Scalable Multimodal Continual Instruction Tuning25 May
- 2025MLLM-CL: Continual Learning for Multimodal Large Language Models5 Jun
Other references · 7
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2017Decoupled Weight Decay Regularizationcited by 1,722
- 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
- 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
- 2022Don't Stop Learning: Towards Continual Learning for the CLIP Modelcited by 61
- 2025PlantExpertVQA: A Visual Question Answering Dataset for Benchmarking Vision-Language Models in Plant Sciencecited by 3
- 2026Beyond Point-wise Neural Collapse: A Topology-Aware Hierarchical Classifier for Class-Incremental Learningcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning
- 2025Routing-Based Continual Learning for Multimodal Large Language Models
- 2026CoRe-MoE: Compact Reusable MoE for Continual Multimodal Instruction Tuning
- 2026CRAM: Centroid-Routing and Adaptive MoE for Multimodal Continual Instruction Tuning
- 2025Enhancing Multimodal Continual Instruction Tuning with BranchLoRA
- 2026StrLoRA: Towards Streaming Continual Visual Instruction Tuning for MLLMs
- 2026PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning
- 2026ProtoAda: Prototype-Guided Adaptive Adapter Expansion and Geometric Consolidation for Multimodal Continual Instruction Tuning