mlsanityData to 14 Oct 2026

On Quadratic Penalties in Elastic Weight Consolidation

arXiv 1712.03847 ↗First posted 11 Dec 2017stat.ML cs.LGcited by 55

Not on the map. Why?

Abstract (arXiv)

Elastic weight consolidation (EWC, Kirkpatrick et al, 2017) is a novel algorithm designed to safeguard against catastrophic forgetting in neural networks. EWC can be seen as an approximation to Laplace propagation (Eskin et al, 2004), and this view is consistent with the motivation given by Kirkpatrick et al (2017). In this note, I present an extended derivation that covers the case when there are more than two tasks. I show that the quadratic penalties in EWC are inconsistent with this derivation and might lead to double-counting data from earlier tasks.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Dec 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 11; 2022: 4; 2023: 5; 2024: 9; 2025: 15; 2026: 11.

55 papers.

Sort:

By area:

  1. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 365
  2. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 116
  3. 2023Model Merging by Uncertainty-Based Gradient Matchingcited by 77
  4. 2021Posterior Meta-Replay for Continual Learningcited by 45
  5. 2024Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancementcited by 45
  6. 2021DualNet: Continual Learning, Fast and Slowcited by 39
  7. 2021Open-world Machine Learning: Applications, Challenges, and Opportunitiescited by 34
  8. 2023Neuro-Symbolic Continual Learning: Knowledge, Reasoning Shortcuts and Concept Rehearsalcited by 33
  9. 2024A Statistical Theory of Regularization-Based Continual Learningcited by 33
  10. 2021CLeaR: An Adaptive Continual Learning Framework for Regression Taskscited by 22
  11. 2021Continual Learning in Sensor-based Human Activity Recognition: an Empirical Benchmark Analysiscited by 21
  12. 2021How do Quadratic Regularizers Prevent Catastrophic Forgetting: The Role of Interpolationcited by 19
38 more
  1. 2024A Second-Order Perspective on Model Compositionality and Incremental Learningcited by 18
  2. 2024Scalable Bayesian Learning with posteriorscited by 13
  3. 2025On the Computation of the Fisher Information in Continual Learningcited by 13
  4. 2024EVCL: Elastic Variational Continual Learning with Weight Consolidationcited by 9
  5. 2025BECAME: BayEsian Continual Learning with Adaptive Model MErgingcited by 9
  6. 2021Stabilizing Elastic Weight Consolidation method in practical ML tasks and using weight importances for neural network pruningcited by 8
  7. 2023Deep Reinforcement Learning for Autonomous Cyber Defence: A Surveycited by 7
  8. 2025Knowledge Retention for Continual Model-Based Reinforcement Learningcited by 7
  9. 2021Towards continual task learning in artificial neural networks: current approaches and insights from neurosciencecited by 6
  10. 2025A Unified Gradient-based Framework for Task-agnostic Continual Learning-Unlearningcited by 6
  11. 2025A Multi-Memory Segment System for Generating High-Quality Long-Term Memory Content in Agentscited by 6
  12. 2023Approximate Bayesian Class-Conditional Models under Continuous Representation Shiftcited by 5
  13. 2024CORE: Mitigating Catastrophic Forgetting in Continual Learning through Cognitive Replaycited by 5
  14. 2024Region-Based Optimization in Continual Learning for Audio Deepfake Detectioncited by 5
  15. 2025Pareto Continual Learning: Preference-Conditioned Learning and Adaption for Dynamic Stability-Plasticity Trade-offcited by 3
  16. 2025Continuous Evolution Pool: Taming Recurring Concept Drift in Online Time Series Forecastingcited by 3
  17. 2026Revisiting Weight Regularization for Low-Rank Continual Learningcited by 3
  18. 2026Information-Theoretic Constraints for Continual Vision-Language-Action Alignmentcited by 3
  19. 2023One-Shot Machine Unlearning with Mnemonic Codecited by 2
  20. 2025Parabolic Continual Learningcited by 2
  21. 2025EFC++: Elastic Feature Consolidation with Prototype Re-balancing for Cold Start Exemplar-free Incremental Learningcited by 2
  22. 2021Improving Scheduled Sampling with Elastic Weight Consolidation for Neural Machine Translationcited by 1
  23. 2022DIRA: Dynamic Domain Incremental Regularised Adaptationcited by 1
  24. 2024Decentralized and Lifelong-Adaptive Multi-Agent Collaborative Learningcited by 1
  25. 2025Lifelong Reinforcement Learning with Similarity-Driven Weighting by Large Modelscited by 1
  26. 2025Continual Learning With Quasi-Newton Methodscited by 1
  27. 2025Adaptive Variance-Penalized Continual Learning with Fisher Regularizationcited by 1
  28. 2026Affect and Effect: Limitations of regularisation-based continual learning in EEG-based emotion classificationcited by 1
  29. 2026Interactionless Inverse Reinforcement Learning: A Data-Centric Framework for Durable Alignmentcited by 1
  30. 2022Empirical investigations on WVA structural issuescited by 0
  31. 2022Continual Learning, Fast and Slowcited by 0
  32. 2022Correlation of the importances of neural network weights calculated by modern methods of overcoming catastrophic forgettingcited by 0
  33. 2024On Sequential Maximum a Posteriori Inference for Continual Learningcited by 0
  34. 2025Sequential Function-Space Variational Inference via Gaussian Mixture Approximationcited by 0
  35. 2025Closing the Oracle Gap: Increment Vector Transformation for Class Incremental Learningcited by 0
  36. 2025Steering Vision-Language Pre-trained Models for Incremental Face Presentation Attack Detectioncited by 0
  37. 2026Continual Quantum Architecture Search with Tensor-Train Encoding: Theory and Applications to Signal Processingcited by 0
  38. 2026Amnesia: A Stealthy Replay Attack on Continual Learning Dreamscited by 0

50 most cited of 55 All 55 (JSON)