mlsanityData to 14 Oct 2026

On the Computation of the Fisher Information in Continual Learning

arXiv 2502.11756 ↗First posted 17 Feb 2025cs.LG cs.AI cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

One of the most popular methods for continual learning with deep neural networks is Elastic Weight Consolidation (EWC), which involves computing the Fisher Information. The exact way in which the Fisher Information is computed is however rarely described, and multiple different implementations for it can be found online. This blog post discusses and empirically compares several often-used implementations, which highlights that many currently reported results for EWC could likely be improved by changing the way the Fisher Information is computed.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 13 of its 20 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

13 of 20 references.

References · 13

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  3. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 742
  4. 2014New insights and perspectives on the natural gradient methodcited by 443
  5. 2019Limitations of the Empirical Fisher Approximation for Natural Gradient Descentcited by 217
  6. 2018Online Structured Laplace Approximations For Overcoming Catastrophic Forgettingcited by 201
  7. 2018Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgettingcited by 127
  8. 2021PyCIL: A Python Toolbox for Class-Incremental Learningcited by 83
  9. 2017On Quadratic Penalties in Elastic Weight Consolidationcited by 55
  10. 2021Natural continual learning: success is a journey, not (just) a destinationcited by 46
  11. 2023Avalanche: A PyTorch Library for Deep Continual Learningcited by 33
  12. 2021Elastic Weight Consolidation (EWC): Nuts and Boltscited by 23
1 more
  1. 2023Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also Howcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 8.

13 papers.

Sort:

By area:

  1. 2025Full-Parameter Continual Pretraining of Gemma2: Insights into Fluency and Domain Knowledgecited by 4
  2. 2025HERMAN: Hierarchical Representation Matching for CLIP-based Class-Incremental Learningcited by 3
  3. 2026Revisiting Weight Regularization for Low-Rank Continual Learningcited by 3
  4. 2026SLE-FNO: Single-Layer Extensions for Task-Agnostic Continual Learning in Fourier Neural Operatorscited by 3
  5. 2025Sparsity-Driven Plasticity in Multi-Task Reinforcement Learningcited by 2
  6. 2026Fisher-Orthogonal Projected Natural Gradient Descent for Continual Learningcited by 2
  7. 2026Self-Consolidating Language Models: Continual Knowledge Incorporation from Contextcited by 2
  8. 2026Affect and Effect: Limitations of regularisation-based continual learning in EEG-based emotion classificationcited by 1
  9. 2025BRAIN: Bias-Mitigation Continual Learning Approach to Vision-Brain Understandingcited by 0
  10. 2025Unbiased Online Curvature Approximation for Regularized Graph Continual Learningcited by 0
  11. 2026Balancing Stability and Plasticity in Sequentially Trained Early-Exiting Neural Networkscited by 0
  12. 2026Normative Loss Landscape Navigation: A Trajectory-Based Approach to Mitigating Forgetting in Incremental Learningcited by 0
1 more
  1. 2026Do Artificial Neural Networks Need Dendritic Nanotubes? Two Reasons for Shrinking Importance Estimates in Continual Learningcited by 0