# InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memories

arXiv: https://arxiv.org/abs/2607.02010 · HTML: https://mlsanity.com/paper/2607.02010/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 22 of its 30 references that are in the map.
- First posted on arXiv 2 Jul 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 22 of 22)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019)
- [COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysis](https://mlsanity.com/paper/1903.02874/) (2019)
- [Memorizing Transformers](https://mlsanity.com/paper/2203.08913/) (2022)
- [SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning](https://mlsanity.com/paper/2505.02486/) (2025)
- [HiDe-LLaVA: Hierarchical Decoupling for Continual Instruction Tuning of Multimodal Large Language Model](https://mlsanity.com/paper/2503.12941/) (2025)
- [Diversity Measurement and Subset Selection for Instruction Tuning Datasets](https://mlsanity.com/paper/2402.02318/) (2024)
- [Dynamic Mixture of Curriculum LoRA Experts for Continual Multimodal Instruction Tuning](https://mlsanity.com/paper/2506.11672/) (2025)
- [Synthetic Data is an Elegant GIFT for Continual Vision-Language Models](https://mlsanity.com/paper/2503.04229/) (2025)
- [Continual Multimodal Contrastive Learning](https://mlsanity.com/paper/2503.14963/) (2025)
- [VQA2: Visual Question Answering for Video Quality Assessment](https://mlsanity.com/paper/2411.03795/) (2024)
- [Enhancing Multimodal Continual Instruction Tuning with BranchLoRA](https://mlsanity.com/paper/2506.02041/) (2025)
- [LADA: Scalable Label-Specific CLIP Adapter for Continual Learning](https://mlsanity.com/paper/2505.23271/) (2025)
- [Speculative Coreset Selection for Task-Specific Fine-tuning](https://mlsanity.com/paper/2410.01296/) (2024)
- [Adapt-∞: Scalable Continual Multimodal Instruction Tuning via Dynamic Data Selection](https://mlsanity.com/paper/2410.10636/) (2024)
- [When Continue Learning Meets Multimodal Large Language Model: A Survey](https://mlsanity.com/paper/2503.01887/) (2025)
- [Mitigating Intra- and Inter-modal Forgetting in Continual Learning of Unified Multimodal Models](https://mlsanity.com/paper/2512.03125/) (2025)
- [Instruction-Grounded Visual Projectors for Continual Learning of Generative Vision-Language Models](https://mlsanity.com/paper/2508.00260/) (2025)
- [Proxy-FDA: Proxy-based Feature Distribution Alignment for Fine-tuning Vision Foundation Models without Forgetting](https://mlsanity.com/paper/2505.24088/) (2025)
- [MIRA: Empowering One-Touch AI Services on Smartphones with MLLM-based Instruction Recommendation](https://mlsanity.com/paper/2509.13773/) (2025)

## Abstract (arXiv)

Multimodal large language models must adapt to evolving tasks and domains, yet continual improvement under bounded deployment footprint remains difficult because repeated parameter updates or growing replay stores can accumulate adaptation state over time. We study fixed-footprint continual adaptation: the deployed adaptation state is kept under a fixed memory budget, while the backbone model is left unchanged and task-specific updates are externalized. We propose InduceKV, a retrieval-based method that stores each selected training prefix as an attention-ready memory entry, consisting of a frozen retrieval key and compact layerwise key--value (KV) payloads that can be appended to the model's self-attention cache. Under a strict memory budget, InduceKV constructs a compact inducing set through bilevel selection: a lightweight calibration is fit for retrieval, while the selected memory balances current-task likelihood, anchor-based retention, and coverage in the frozen retrieval space. Across task-incremental instruction tuning, continual VQA, domain-incremental adaptation, and lifelong multimodal instruction tuning, InduceKV consistently improves over PEFT, MoE, replay, and prompt-retrieval baselines under matched memory budgets. We further report backbone-matched, stage-1 CoIN, compute-matched, and scalability diagnostics, showing that the gains are not due to a stronger backbone, replay alone, or an unbounded candidate pool.