mlsanityData to 7 Oct 2026

On Continual Model Refinement in Out-of-Distribution Data Streams

arXiv 2205.02014 ↗First posted 4 May 2022cs.CL cs.AI cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

Real-world natural language processing (NLP) models need to be continually updated to fix the prediction errors in out-of-distribution (OOD) data streams while overcoming catastrophic forgetting. However, existing continual learning (CL) problem setups cannot cover such a realistic and complex scenario. In response to this, we propose a new CL problem formulation dubbed continual model refinement (CMR). Compared to prior CL settings, CMR is more practical and introduces unique challenges (boundary-agnostic and non-stationary distribution shift, diverse mixtures of multiple OOD data clusters, error-centric streams, etc.). We extend several existing CL approaches to the CMR setting and evaluate them extensively. For benchmarking and analysis, we propose a general sampling algorithm to obtain dynamic OOD data streams with controllable non-stationarity, as well as a suite of metrics measuring various aspects of online performance. Our experiments and detailed analysis reveal the promise and challenges of the CMR problem, supporting that studying CMR in dynamic OOD streams can benefit the longevity of deployed NLP models in production.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 26 of its 31 references that are in the map. First posted on arXiv 4 May 2022.

Builds on

26 of 31 references.

References · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  8. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  9. 2018Experience Replay for Continual Learningcited by 1,074
  10. 2021Editing Factual Knowledge in Language Modelscited by 548
  11. 2021Fast Model Editing at Scalecited by 491
  12. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 376
14 more
  1. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  2. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  3. 2020Modifying Memories in Transformer Modelscited by 222
  4. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 221
  5. 2019MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehensioncited by 183
  6. 2021Towards Continual Knowledge Learning of Language Modelscited by 172
  7. 2019LAMOL: LAnguage MOdeling for Lifelong Language Learningcited by 156
  8. 2020Continual Lifelong Learning in Natural Language Processing: A Surveycited by 152
  9. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  10. 2019Accelerating Deep Learning by Focusing on the Biggest Loserscited by 94
  11. 2021Continual Learning for Text Classification with Information Disentanglement Based Regularizationcited by 86
  12. 2019Sentence Embedding Alignment for Lifelong Relation Extractioncited by 70
  13. 2019Ordered SGD: A New Stochastic Optimization Framework for Empirical Risk Minimizationcited by 51
  14. 2021Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learningcited by 35

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 7; 2024: 10; 2025: 0; 2026: 1.

22 papers.

Sort:

By area:

  1. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 267
  2. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  3. 2022Fine-tuned Language Models are Continual Learnerscited by 152
  4. 2022Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Timecited by 85
  5. 2023MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRAcited by 80
  6. 2023Lifelong Language Pretraining with Distribution-Specialized Expertscited by 64
  7. 2023Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curationcited by 38
  8. 2023How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advancescited by 33
  9. 2024Data-driven Discovery with Large Generative Modelscited by 33
  10. 2022Challenging Common Assumptions about Catastrophic Forgettingcited by 20
  11. 2024LEMoE: Advanced Mixture of Experts Adaptor for Lifelong Model Editing of Large Language Modelscited by 14
  12. 2024ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Taskscited by 11
10 more
  1. 2024What Will My Model Forget? Forecasting Forgotten Examples in Language Model Refinementcited by 9
  2. 2023Robust Question Answering against Distribution Shifts with Test-Time Adaptation: An Empirical Studycited by 8
  3. 2024MoE-CT: A Novel Approach For Large Language Models Training With Resistance To Catastrophic Forgettingcited by 6
  4. 2024Scalable Model Editing via Customized Expert Networkscited by 5
  5. 2024Gradient Localization Improves Lifelong Pretraining of Language Modelscited by 3
  6. 2024Novel-WD: Exploring acquisition of Novel World Knowledge in LLMs Using Prefix-Tuningcited by 2
  7. 2023Estimating Semantic Similarity between In-Domain and Out-of-Domain Samplescited by 1
  8. 2024Better Late Than Never: Formulating and Benchmarking Recommendation Editingcited by 1
  9. 2026Reversible Lifelong Model Editing via Semantic Routing-Based LoRAcited by 1
  10. 2023Continual Learning for Out-of-Distribution Pedestrian Detectioncited by 0