mlsanityData to 7 Oct 2026

InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memories

arXiv 2607.02010 ↗First posted 2 Jul 2026cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Multimodal large language models must adapt to evolving tasks and domains, yet continual improvement under bounded deployment footprint remains difficult because repeated parameter updates or growing replay stores can accumulate adaptation state over time. We study fixed-footprint continual adaptation: the deployed adaptation state is kept under a fixed memory budget, while the backbone model is left unchanged and task-specific updates are externalized. We propose InduceKV, a retrieval-based method that stores each selected training prefix as an attention-ready memory entry, consisting of a frozen retrieval key and compact layerwise key--value (KV) payloads that can be appended to the model's self-attention cache. Under a strict memory budget, InduceKV constructs a compact inducing set through bilevel selection: a lightweight calibration is fit for retrieval, while the selected memory balances current-task likelihood, anchor-based retention, and coverage in the frozen retrieval space. Across task-incremental instruction tuning, continual VQA, domain-incremental adaptation, and lifelong multimodal instruction tuning, InduceKV consistently improves over PEFT, MoE, replay, and prompt-retrieval baselines under matched memory budgets. We further report backbone-matched, stage-1 CoIN, compute-matched, and scalability diagnostics, showing that the gains are not due to a stronger backbone, replay alone, or an unbounded candidate pool.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 22 of its 30 references that are in the map. First posted on arXiv 2 Jul 2026.

Builds on

22 of 30 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Instruction-Grounded Visual Projectors for Continual Learning of Generative Vision-Language Models1 Aug
  2. 2025Dynamic Mixture of Curriculum LoRA Experts for Continual Multimodal Instruction Tuning13 Jun
  3. 2025Enhancing Multimodal Continual Instruction Tuning with BranchLoRA31 May
  4. 2025LADA: Scalable Label-Specific CLIP Adapter for Continual Learning29 May
  5. 2025SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning5 May
  6. 2025Synthetic Data is an Elegant GIFT for Continual Vision-Language Models6 Mar
  7. 2025When Continue Learning Meets Multimodal Large Language Model: A Survey27 Feb

Other references · 15

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  3. 2023A Survey on Multimodal Large Language Modelscited by 898
  4. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  5. 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 316
  6. 2022Memorizing Transformerscited by 219
  7. 2025HiDe-LLaVA: Hierarchical Decoupling for Continual Instruction Tuning of Multimodal Large Language Modelcited by 29
  8. 2024Diversity Measurement and Subset Selection for Instruction Tuning Datasetscited by 28
  9. 2025Continual Multimodal Contrastive Learningcited by 19
  10. 2024VQA2: Visual Question Answering for Video Quality Assessmentcited by 15
  11. 2024Speculative Coreset Selection for Task-Specific Fine-tuningcited by 10
  12. 2024Adapt-∞: Scalable Continual Multimodal Instruction Tuning via Dynamic Data Selectioncited by 9
3 more
  1. 2025Mitigating Intra- and Inter-modal Forgetting in Continual Learning of Unified Multimodal Modelscited by 6
  2. 2025Proxy-FDA: Proxy-based Feature Distribution Alignment for Fine-tuning Vision Foundation Models without Forgettingcited by 1
  3. 2025MIRA: Empowering One-Touch AI Services on Smartphones with MLLM-based Instruction Recommendationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Beyond Anti-Forgetting: Multimodal Continual Instruction Tuning with Positive Forward Transfer
  2. 2026CRAM: Centroid-Routing and Adaptive MoE for Multimodal Continual Instruction Tuning
  3. 2026Is Our Benchmark Enough? An Analysis of Continual Learning for MLLMs
  4. 2025Multimodal Continual Instruction Tuning with Dynamic Gradient Guidance
  5. 2025Routing-Based Continual Learning for Multimodal Large Language Models
  6. 2026CoRe-MoE: Compact Reusable MoE for Continual Multimodal Instruction Tuning
  7. 2026MAny: Merge Anything for Multimodal Continual Instruction Tuning
  8. 2024Continual LLaVA: Continual Instruction Tuning in Large Vision-Language Models