mlsanityData to 14 Oct 2026

Continual Learning Using Only Large Language Model Prompting

arXiv 2412.15479 ↗First posted 20 Dec 2024cs.CL cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

We introduce CLOB, a novel continual learning (CL) paradigm wherein a large language model (LLM) is regarded as a black box. Learning is done incrementally via only verbal prompting. CLOB does not fine-tune any part of the LLM or add any trainable parameters to it. It is particularly suitable for LLMs that are accessible via APIs. We also propose a new CL technique, called CIS, based on incremental summarization that also overcomes the LLM's input length limit. Experiments show CIS outperforms baselines by a very large margin.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 29 of its 41 references that are in the map. First posted on arXiv 20 Dec 2024.

Builds on

29 of 41 references.

References · 29

  1. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  4. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  5. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 681
  6. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  7. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
  8. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 433
  9. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 386
  10. 2023Continual Pre-training of Language Modelscited by 198
  11. 2019Episodic Memory in Lifelong Language Learningcited by 188
  12. 2021DEMix Layers: Disentangling Domains for Modular Language Modelingcited by 132
17 more
  1. 2021Achieving Forgetting Prevention and Knowledge Transfer in Continual Learningcited by 91
  2. 2021Continual Learning for Text Classification with Information Disentanglement Based Regularizationcited by 86
  3. 2020Continual Learning in Task-Oriented Dialogue Systemscited by 83
  4. 2022Continual Learning of Natural Language Processing Tasks: A Surveycited by 83
  5. 2022A Theoretical Study on Solving Continual Learningcited by 80
  6. 2022ELLE: Efficient Lifelong Pre-training for Emerging Datacited by 70
  7. 2021Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inferencecited by 67
  8. 2024InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructionscited by 58
  9. 2020Efficient Meta Lifelong-Learning with Limited Memorycited by 48
  10. 2024Why Larger Language Models Do In-context Learning Differently?cited by 45
  11. 2022Continual Prompt Tuning for Dialog State Trackingcited by 44
  12. 2023Learnability and Algorithm for Continual Learningcited by 28
  13. 2024Rehearsal-Free Modular and Compositional Continual Learning for Language Modelscited by 27
  14. 2021Continual Learning for Task-oriented Dialogue System with Iterative Network Pruning, Expanding and Maskingcited by 22
  15. 2021Lifelong Intent Detection via Multi-Strategy Rebalancingcited by 12
  16. 2023Class-Incremental Learning based on Label Generationcited by 11
  17. 2024Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language Learningcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2025MetaFaith: Faithful Natural Language Uncertainty Expression in LLMscited by 16
  2. 2026Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stagescited by 6
  3. 2026RIZZ: Routing Interactions to Near Zero-Interference Zones for Continual Adaptation of Black-Box Agentscited by 1
  4. 2026MIITA: Memory-Induced Inference-Time Adaptation for Continual Learning with Small Language Modelscited by 0
  5. 2026Learning an Anchored Prompt Space for Continual Adaptation of Large Language Modelscited by 0