mlsanityData to 14 Oct 2026

Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence

arXiv 1801.10112 ↗First posted 30 Jan 2018cs.CVcited by 742

Not on the map. Why?

Cited by 27 in Knowledge distillation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Incremental learning (IL) has received a lot of attention recently, however, the literature lacks a precise problem definition, proper evaluation settings, and metrics tailored specifically for the IL problem. One of the main objectives of this work is to fill these gaps so as to provide a common ground for better understanding of IL. The main challenge for an IL algorithm is to update the classifier whilst preserving existing knowledge. We observe that, in addition to forgetting, a known issue while preserving knowledge, IL also suffers from a problem we call intransigence, inability of a model to update its knowledge. We introduce two metrics to quantify forgetting and intransigence that allow us to understand, analyse, and gain better insights into the behaviour of IL algorithms. We present RWalk, a generalization of EWC++ (our efficient version of EWC [Kirkpatrick2016EWC]) and Path Integral [Zenke2017Continual] with a theoretically grounded KL-divergence based perspective. We provide a thorough analysis of various IL algorithms on MNIST and CIFAR-100 datasets. In these experiments, RWalk obtains superior results in terms of accuracy, and also provides a better trade-off between forgetting and intransigence.

Key facts. Cited by 742 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Jan 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 102; 2022: 127; 2023: 149; 2024: 158; 2025: 135; 2026: 71.

742 papers.

Sort:

By area:

  1. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  2. 2021Learning to Prompt for Continual Learningcited by 944
  3. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  4. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  5. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  6. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  7. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 245
  8. 2021Co2L: Contrastive Continual Learningcited by 244
  9. 2023Class-Incremental Learning: A Surveycited by 209
  10. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 184
  11. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 176
  12. 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 164
38 more
  1. 2021Self-Supervised Models are Continual Learnerscited by 149
  2. 2021Training Networks in Null Space of Feature Covariance for Continual Learningcited by 143
  3. 2021Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learningcited by 141
  4. 2021An Empirical Investigation of the Role of Pre-training in Lifelong Learningcited by 134
  5. 2022Continual Learning and Private Unlearningcited by 130
  6. 2022Class-Incremental Continual Learning into the eXtended DER-versecited by 124
  7. 2021Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minimacited by 118
  8. 2022Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learningcited by 118
  9. 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
  10. 2021Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learningcited by 115
  11. 2024Higher Layers Need More LoRA Expertscited by 102
  12. 2024LLaMA Pro: Progressive LLaMA with Block Expansioncited by 101
  13. 2021Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representationscited by 96
  14. 2023Fine-tuning can cripple your foundation model; preserving features may be the solutioncited by 96
  15. 2021GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learningcited by 95
  16. 2022The CLEAR Benchmark: Continual LEArning on Real-World Imagerycited by 93
  17. 2022Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Timecited by 90
  18. 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
  19. 2021Continual Learning for Text Classification with Information Disentanglement Based Regularizationcited by 86
  20. 2021On Learning the Geodesic Path for Incremental Learningcited by 85
  21. 2022Continual Learning of Natural Language Processing Tasks: A Surveycited by 83
  22. 2022TRGP: Trust Region Gradient Projection for Continual Learningcited by 82
  23. 2022Memory Replay with Data Compression for Continual Learningcited by 82
  24. 2023Learning without Forgetting for Vision-Language Modelscited by 79
  25. 2021AFEC: Active Forgetting of Negative Transfer in Continual Learningcited by 73
  26. 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 72
  27. 2022Representation Compensation Networks for Continual Semantic Segmentationcited by 72
  28. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 69
  29. 2021ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learningcited by 68
  30. 2022CLiMB: A Continual Learning Benchmark for Vision-and-Language Taskscited by 68
  31. 2021Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inferencecited by 67
  32. 2023Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifierscited by 67
  33. 2023TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Modelscited by 66
  34. 2023TARGET: Federated Class-Continual Learning via Exemplar-Free Distillationcited by 65
  35. 2023Maintaining Plasticity in Continual Learning via Regenerative Regularizationcited by 65
  36. 2023IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturingcited by 64
  37. 2023Computationally Budgeted Continual Learning: What Does Matter?cited by 61
  38. 2023Decoupled Kullback-Leibler Divergence Losscited by 58

50 most cited of 742 All 742 (JSON)