mlsanityData to 14 Oct 2026

Plug-and-Play Adaptation for Continuously-updated QA

arXiv 2204.12785 ↗First posted 27 Apr 2022cs.CLcited by 21

Not on the map. Why?

Abstract (arXiv)

Language models (LMs) have shown great potential as implicit knowledge bases (KBs). And for their practical use, knowledge in LMs need to be updated periodically. However, existing tasks to assess LMs' efficacy as KBs do not adequately consider multiple large-scale updates. To this end, we first propose a novel task--Continuously-updated QA (CuQA)--in which multiple large-scale updates are made to LMs, and the performance is measured with respect to the success in adding and updating knowledge while retaining existing knowledge. We then present LMs with plug-in modules that effectively handle the updates. Experiments conducted on zsRE QA and NQ datasets show that our method outperforms existing approaches. We find that our method is 4x more effective in terms of updates/forgets ratio, compared to a fine-tuning baseline.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 22 of its 24 references that are in the map. First posted on arXiv 27 Apr 2022.

Builds on

22 of 24 references.

References · 22

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  6. 2019Language Models as Knowledge Bases?cited by 1,936
  7. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  8. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  9. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  10. 2021Editing Factual Knowledge in Language Modelscited by 553
  11. 2021Fast Model Editing at Scalecited by 498
  12. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
10 more
  1. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
  2. 2020K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapterscited by 319
  3. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 296
  4. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 214
  5. 2020Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgettingcited by 170
  6. 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
  7. 2020Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learningcited by 139
  8. 2020Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queriescited by 107
  9. 2019Augmenting Self-attention with Persistent Memorycited by 93
  10. 2021Efficient Passage Retrieval with Hashing for Open-domain Question Answeringcited by 53

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 8; 2025: 4; 2026: 3.

21 papers.

Sort:

By area:

  1. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
  2. 2022RealTime QA: What's the Answer Right Now?cited by 244
  3. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  4. 2023Knowledge Editing for Large Language Models: A Surveycited by 194
  5. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 83
  6. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editingcited by 79
  7. 2024Towards Lifelong Learning of Large Language Models: A Surveycited by 64
  8. 2024In-Context Editing: Learning Knowledge from Self-Induced Distributionscited by 34
  9. 2023How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advancescited by 33
  10. 2023Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Modelscited by 32
  11. 2023Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Modelscited by 15
  12. 2024Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Modelscited by 14
9 more
  1. 2025DELMAN: Dynamic Defense Against Large Language Model Jailbreaking with Model Editingcited by 10
  2. 2024Towards Practical Tool Usage for Continually Learning LLMscited by 6
  3. 2026Investigating Tool-Memory Conflicts in Tool-Augmented LLMscited by 6
  4. 2025Continuously Steering LLMs Sensitivity to Contextual Knowledge with Proxy Modelscited by 5
  5. 2024Semantic are Beacons: A Semantic Perspective for Unveiling Parameter-Efficient Fine-Tuning in Knowledge Learningcited by 3
  6. 2025Explaining Black-box Language Models with Knowledge Probing Systems: A Post-hoc Explanation Perspectivecited by 1
  7. 2026Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inferencecited by 1
  8. 2024SPA: Towards A Computational Friendly Cloud-Base and On-Devices Collaboration Seq2seq Personalized Generation with Casual Inferencecited by 0
  9. 2026EVA: Editing for Versatile Alignment against Jailbreakscited by 0