mlsanityData to 14 Oct 2026

Gradient Episodic Memory for Continual Learning

arXiv 1706.08840 ↗First posted 26 Jun 2017cs.LG cs.AIcited by 1,898

Not on the map. Why?

Cited by 51 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

One major obstacle towards AI is the poor ability of models to solve new problems quicker, and without forgetting previously acquired knowledge. To better understand this issue, we study the problem of continual learning, where the model observes, once and one by one, examples concerning a sequence of tasks. First, we propose a set of metrics to evaluate models learning over a continuum of data. These metrics characterize models not only by their test accuracy, but also in terms of their ability to transfer knowledge across tasks. Second, we propose a model for continual learning, called Gradient Episodic Memory (GEM) that alleviates forgetting, while allowing beneficial transfer of knowledge to previous tasks. Our experiments on variants of the MNIST and CIFAR-100 datasets demonstrate the strong performance of GEM when compared to the state-of-the-art.

Key facts. Cited by 1,898 papers in the map (data through 14 Oct 2026). First posted on arXiv 26 Jun 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 235; 2022: 288; 2023: 336; 2024: 338; 2025: 319; 2026: 382.

1,898 papers.

Sort:

By area:

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  2. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  3. 2021Learning to Prompt for Continual Learningcited by 944
  4. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  5. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 642
  6. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  7. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 450
  8. 2022Memory-Based Model Editing at Scalecited by 447
  9. 2021Gradient Projection Memory for Continual Learningcited by 348
  10. 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 312
  11. 2021Towards Open World Object Detectioncited by 297
  12. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
38 more
  1. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  2. 2021Gradient Matching for Domain Generalizationcited by 258
  3. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  4. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 245
  5. 2021Co2L: Contrastive Continual Learningcited by 244
  6. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
  7. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 230
  8. 2021New Insights on Reducing Abrupt Representation Change in Online Continual Learningcited by 219
  9. 2023Class-Incremental Learning: A Surveycited by 209
  10. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
  11. 2023Progressive Prompts: Continual Learning for Language Modelscited by 197
  12. 2022Patching open-vocabulary models by interpolating weightscited by 190
  13. 2021Fishr: Invariant Gradient Variances for Out-of-Distribution Generalizationcited by 186
  14. 2021Generalized Source-free Domain Adaptationcited by 175
  15. 2021Towards Continual Knowledge Learning of Language Modelscited by 174
  16. 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 164
  17. 2025Dynamic Cheatsheet: Test-Time Learning with Adaptive Memorycited by 163
  18. 2022A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learningcited by 154
  19. 2021Self-Supervised Models are Continual Learnerscited by 149
  20. 2024Continual Learning for Large Language Models: A Surveycited by 147
  21. 2021Training Networks in Null Space of Feature Covariance for Continual Learningcited by 143
  22. 2021Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learningcited by 141
  23. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRAcited by 141
  24. 2021An Empirical Investigation of the Role of Pre-training in Lifelong Learningcited by 134
  25. 2021DEMix Layers: Disentangling Domains for Modular Language Modelingcited by 132
  26. 2021Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learningcited by 131
  27. 2022TimeLMs: Diachronic Language Models from Twittercited by 131
  28. 2022Continual Learning and Private Unlearningcited by 130
  29. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 130
  30. 2026MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memorycited by 129
  31. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 125
  32. 2021Avalanche: an End-to-End Library for Continual Learningcited by 124
  33. 2022Class-Incremental Continual Learning into the eXtended DER-versecited by 124
  34. 2023EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularizationcited by 123
  35. 2024Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsalcited by 118
  36. 2021Online Coreset Selection for Rehearsal-based Continual Learningcited by 117
  37. 2021AutoNovel: Automatically Discovering and Learning Novel Visual Categoriescited by 116
  38. 2022Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning Systemcited by 112

50 most cited of 1,898 All 1,898 (JSON)