mlsanityData to 7 Oct 2026

I2I: Initializing Adapters with Improvised Knowledge

arXiv 2304.02168 ↗First posted 4 Apr 2023cs.CL cs.AI cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Adapters present a promising solution to the catastrophic forgetting problem in continual learning. However, training independent Adapter modules for every new task misses an opportunity for cross-task knowledge transfer. We propose Improvise to Initialize (I2I), a continual learning algorithm that initializes Adapters for incoming tasks by distilling knowledge from previously-learned tasks' Adapters. We evaluate I2I on CLiMB, a multimodal continual learning benchmark, by conducting experiments on sequences of visual question answering tasks. Adapters trained with I2I consistently achieve better task accuracy than independently-trained Adapters, demonstrating that our algorithm facilitates knowledge transfer between task Adapters. I2I also results in better cross-task knowledge transfer than the state-of-the-art AdapterFusion without incurring the associated parametric cost.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 25 of its 30 references that are in the map. First posted on arXiv 4 Apr 2023.

Builds on

25 of 30 references.

References · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  5. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  6. 2015VQA: Visual Question Answeringcited by 2,489
  7. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  8. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  9. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  10. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  11. 2017Learning multiple visual domains with residual adapterscited by 553
  12. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 319
13 more
  1. 2015Visual7W: Grounded Question Answering in Imagescited by 315
  2. 2019Continual learning with hypernetworkscited by 267
  3. 2018The Natural Language Decathlon: Multitask Learning as Question Answeringcited by 235
  4. 2020Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?cited by 127
  5. 2014A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Inputcited by 112
  6. 2022CLiMB: A Continual Learning Benchmark for Vision-and-Language Taskscited by 68
  7. 2022Memory Efficient Continual Learning with Transformerscited by 43
  8. 2021Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learningcited by 35
  9. 2022Symbolic Replay: Scene Graph as Prompt for Continual Learning on VQA Taskcited by 35
  10. 2019Psycholinguistics meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answeringcited by 33
  11. 2022Continual Sequence Generation with Adaptive Compositional Modulescited by 31
  12. 2022Continual Learning for Instruction Following from Realtime Feedbackcited by 21
  13. 2022CL-CrossVQA: A Continual Learning Benchmark for Cross-Domain Visual Question Answeringcited by 6

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 4; 2025: 1.

7 papers.

  1. 2024CLAP4CLIP: Continual Learning with Probabilistic Finetuning for Vision-Language Modelscited by 25
  2. 2024Dynamic Transformer Architecture for Continual Learning of Multimodal Taskscited by 8
  3. 2025Merge then Realign: Simple and Effective Modality-Incremental Continual Learning for Multimodal LLMscited by 6
  4. 2023Improved Region Proposal Network for Enhanced Few-Shot Object Detectioncited by 5
  5. 2024Cross-domain Multi-modal Few-shot Object Detection via Rich Textcited by 5
  6. 2024Remembering Transformer for Continual Learningcited by 1
  7. 2023Robust Internal Representations for Domain Generalizationcited by 0