mlsanityData to 14 Oct 2026

Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting

arXiv 1802.02950 ↗First posted 8 Feb 2018cs.CVcited by 127

Not on the map. Why?

Abstract (arXiv)

In this paper we propose an approach to avoiding catastrophic forgetting in sequential task learning scenarios. Our technique is based on a network reparameterization that approximately diagonalizes the Fisher Information Matrix of the network parameters. This reparameterization takes the form of a factorized rotation of parameter space which, when used in conjunction with Elastic Weight Consolidation (which assumes a diagonal Fisher Information Matrix), leads to significantly better performance on lifelong learning of sequential tasks. Experimental results on the MNIST, CIFAR-100, CUB-200 and Stanford-40 datasets demonstrate that we significantly improve the results of standard elastic weight consolidation, and that we obtain competitive results when compared to other state-of-the-art in lifelong learning without forgetting.

Key facts. Cited by 127 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 28; 2022: 18; 2023: 23; 2024: 22; 2025: 28; 2026: 8.

127 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  2. 2021Merging Models with Fisher-Weighted Averagingcited by 621
  3. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  4. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 245
  5. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  6. 2021Rehearsal revealed: The limits and merits of revisiting samples in continual learningcited by 76
  7. 2021Few-Shot Lifelong Learningcited by 59
  8. 2022ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protectioncited by 57
  9. 2021Recent Advances of Continual Learning in Computer Vision: An Overviewcited by 55
  10. 2023Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learningcited by 55
  11. 2022An Efficient Domain-Incremental Learning Approach to Drive in All Weather Conditionscited by 54
  12. 2021Continually Learning Self-Supervised Representations with Projected Functional Regularizationcited by 38
38 more
  1. 2024Elastic Feature Consolidation for Cold Start Exemplar-Free Incremental Learningcited by 37
  2. 2022Long-Tailed Class Incremental Learningcited by 36
  3. 2023The Ideal Continual Learner: An Agent That Never Forgetscited by 35
  4. 2021Curriculum-Meta Learning for Order-Robust Continual Relation Extractioncited by 34
  5. 2021Open-world Machine Learning: Applications, Challenges, and Opportunitiescited by 34
  6. 2022Prototype-Guided Continual Adaptation for Class-Incremental Unsupervised Domain Adaptationcited by 33
  7. 2023Continual Learning via Sequential Function-Space Variational Inferencecited by 29
  8. 2022Consistent Representation Learning for Continual Relation Extractioncited by 28
  9. 2024DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learningcited by 28
  10. 2025CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learningcited by 28
  11. 2023Towards Continual Egocentric Activity Recognition: A Multi-modal Egocentric Activity Dataset for Continual Learningcited by 26
  12. 2024OrCo: Towards Better Generalization via Orthogonality and Contrast for Few-Shot Class-Incremental Learningcited by 26
  13. 2023ICICLE: Interpretable Class Incremental Continual Learningcited by 24
  14. 2023Fixed Design Analysis of Regularization-Based Continual Learningcited by 24
  15. 2023Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomycited by 24
  16. 2022Universal Representations: A Unified Look at Multiple Task and Domain Learningcited by 23
  17. 2025Purifying Shampoo: Investigating Shampoo's Heuristics by Decomposing its Preconditionercited by 23
  18. 2024Generative Multi-modal Models are Good Class-Incremental Learnerscited by 22
  19. 2021Continual Learning in Sensor-based Human Activity Recognition: an Empirical Benchmark Analysiscited by 21
  20. 2023InstructAlign: High-and-Low Resource Language Alignment via Continual Crosslingual Instruction Tuningcited by 21
  21. 2025Continual Multimodal Contrastive Learningcited by 20
  22. 2022Continual Prune-and-Select: Class-incremental learning with specialized subnetworkscited by 17
  23. 2023Online Class-Incremental Learning For Real-World Food Image Classificationcited by 17
  24. 2025Upweighting Easy Samples in Fine-Tuning Mitigates Forgettingcited by 16
  25. 2021Lifelong Learning with Sketched Structural Regularizationcited by 15
  26. 2021Provable Continual Learning via Sketched Jacobian Approximationscited by 15
  27. 2023Continual Multimodal Knowledge Graph Constructioncited by 15
  28. 2023Class Incremental Learning with Pre-trained Vision-Language Modelscited by 15
  29. 2021Lifelong Learning of Hate Speech Classification on Social Mediacited by 14
  30. 2023Plasticity-Optimized Complementary Networks for Unsupervised Continual Learningcited by 14
  31. 2024Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learningcited by 14
  32. 2024Provable Contrastive Continual Learningcited by 13
  33. 2025On the Computation of the Fisher Information in Continual Learningcited by 13
  34. 2023NAPA-VQ: Neighborhood Aware Prototype Augmentation with Vector Quantization for Continual Learningcited by 12
  35. 2024Continual Learning on Graphs: A Surveycited by 12
  36. 2022Continual Learning with Guarantees via Weight Interval Constraintscited by 11
  37. 2022DLCFT: Deep Linear Continual Fine-Tuning for General Incremental Learningcited by 11
  38. 2023Continual Evidential Deep Learning for Out-of-Distribution Detectioncited by 11

50 most cited of 127 All 127 (JSON)