mlsanityData to 14 Oct 2026

Overcoming catastrophic forgetting in neural networks

arXiv 2507.10485 ↗First posted 14 Jul 2025cs.LG cs.IRcited by 55

Not on the map. Why?

Abstract (arXiv)

Catastrophic forgetting is the primary challenge that hinders continual learning, which refers to a neural network ability to sequentially learn multiple tasks while retaining previously acquired knowledge. Elastic Weight Consolidation, a regularization-based approach inspired by synaptic consolidation in biological neural systems, has been used to overcome this problem. In this study prior research is replicated and extended by evaluating EWC in supervised learning settings using the PermutedMNIST and RotatedMNIST benchmarks. Through systematic comparisons with L2 regularization and stochastic gradient descent (SGD) without regularization, we analyze how different approaches balance knowledge retention and adaptability. Our results confirm what was shown in previous research, showing that EWC significantly reduces forgetting compared to naive training while slightly compromising learning efficiency on new tasks. Moreover, we investigate the impact of dropout regularization and varying hyperparameters, offering insights into the generalization of EWC across diverse learning scenarios. These results underscore EWC's potential as a viable solution for lifelong learning in neural networks.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). Builds on 2 of its 3 references that are in the map. First posted on arXiv 14 Jul 2025.

Builds on

2 of 3 references.

References · 2

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 18; 2026: 37.

55 papers.

Sort:

By area:

  1. 2025Continual Learning for VLMs: A Survey and Taxonomy Beyond Forgettingcited by 18
  2. 2025Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruningcited by 13
  3. 2025Forgetful but Faithful: A Cognitive Memory Architecture and Benchmark for Privacy-Aware Generative Agentscited by 13
  4. 2025AIMMerging: Adaptive Iterative Model Merging Using Training Trajectories for Language Model Continual Learningcited by 5
  5. 2026SmartSearch: How Ranking Beats Structure for Conversational Memory Retrievalcited by 5
  6. 2026Geometry Conflict: Explaining and Controlling Forgetting in LLM Continual Post-Trainingcited by 5
  7. 2026Weight Decay Improves Language Model Plasticitycited by 4
  8. 2025Diagnosing the Performance Trade-off in Moral Alignment: A Case Study on Gender Stereotypescited by 2
  9. 2025Pruning as Regularization: Sensitivity-Aware One-Shot Pruning in ASRcited by 2
  10. 2026MedCL-Bench: Benchmarking stability-efficiency trade-offs and scaling in biomedical continual learningcited by 2
  11. 2025KEEP: Integrating Medical Ontologies with Clinical Data for Robust Code Embeddingscited by 1
  12. 2025Privacy-Aware Continual Self-Supervised Learning on Multi-Window Chest Computed Tomography for Domain-Shift Robustnesscited by 1
38 more
  1. 2026Sparse Adapter Fusion for Continual Learning in NLPcited by 1
  2. 2026Learning Hierarchical Orthogonal Prototypes for Generalized Few-Shot 3D Point Cloud Segmentationcited by 1
  3. 2026Brainstacks: Cross-Domain Cognitive Capabilities via Frozen MoE-LoRA Stacks for Continual LLM Learningcited by 1
  4. 2026SCNO: Spiking Compositional Neural Operator -- Towards a Neuromorphic Foundation Model for Nuclear PDE Solvingcited by 1
  5. 2026How Tokenization Limits Phonological Knowledge Representation in Language Models and How to Improve Themcited by 1
  6. 2026Same Methods, Different Rankings: Trainable Depth as an Evaluation Variable in Continual Learningcited by 1
  7. 2026CMI-Mem: Toward Generalizable Long-Term Memory Management via CMI-Augmented Reinforcement Learningcited by 1
  8. 2026Verifier-Induced Support Reshaping in On-Policy Optimizationcited by 1
  9. 2026Learning to Adapt Cross-Domain Preferences via Meta-LoRA for LLM Personalizationcited by 1
  10. 2026AIM: Anchor Identity Features, Then Match for Multimodal Large Language Model Unlearningcited by 1
  11. 2026Where Identity Lives: Localized, Retain-Free Identity Unlearning in Multimodal Large Language Modelscited by 1
  12. 2025Learning from Oblivion: Predicting Knowledge Overflowed Weights via Retrodiction of Forgettingcited by 0
  13. 2025Investigation of D-Wave quantum annealing for training Restricted Boltzmann Machines and mitigating catastrophic forgettingcited by 0
  14. 2025BRAIN: Bias-Mitigation Continual Learning Approach to Vision-Brain Understandingcited by 0
  15. 2025Experience Scaling: Post-Deployment Evolution For Large Language Modelscited by 0
  16. 2025On the Occurence of Critical Learning Periods in Neural Networkscited by 0
  17. 2025Adaptive transfer learning for surgical tool presence detection in laparoscopic videos through gradual freezing fine-tuningcited by 0
  18. 2025ProDER: A Continual Learning Approach for Fault Classification and Localization in Evolving Smart Gridscited by 0
  19. 2025Source-Optimal Training is Transfer-Suboptimalcited by 0
  20. 2025OpenCML: End-to-End Framework of Open-world Machine Learning to Learn Unknown Classes Incrementallycited by 0
  21. 2025Navigating the Reality Gap: On-Device Continual Adaptation of ASR for Clinical Telephonycited by 0
  22. 2026MiLorE-SSL: Scaling Multilingual Capabilities in Self-Supervised Models without Forgettingcited by 0
  23. 2026When Classes Evolve: A Benchmark and Framework for Stage-Aware Class-Incremental Learningcited by 0
  24. 2026Accelerating Stroke MRI with Diffusion Probabilistic Models through Large-Scale Pre-training and Target-Specific Fine-Tuningcited by 0
  25. 2026FedProxy: Federated Fine-Tuning of LLMs via Proxy SLMs and Heterogeneity-Aware Fusioncited by 0
  26. 2026Regression Test Selection for Updated Capability Modules in Compositional ML Systems via Atomic-Quality Probescited by 0
  27. 2026ARGUS: Policy-Adaptive Ad Governance via Evolving Reinforcement with Adversarial Umpiringcited by 0
  28. 2026MANGO: Meta-Adaptive Network Gradient Optimization for Online Continual Learningcited by 0
  29. 2026GoTTA be Diverse: Rethinking Memory Policies for Test-Time Adaptationcited by 0
  30. 2026Retrievable Gradients: Continual Post-Training Without Cumulative Weight Driftcited by 0
  31. 2026Few-Shot Domain Incremental Learning via Continual Vision-Language Consolidationcited by 0
  32. 2026TypiCore: A Hybrid Active Query Strategy for Class-Incremental Learning on Time Seriescited by 0
  33. 2026DiagLoop: A Counterfactual Data Flywheel with Stage-Localized Reinforcement for Diagnostic LLMscited by 0
  34. 2026NeuPAT: Neuron-aware Plasticity Allocation Tuning for Language-Preserving MLLMscited by 0
  35. 2026What to Preserve, Where to Adapt: A Depth-Wise Analysis of Forgetting in Continual Gynecological Image Segmentationcited by 0
  36. 2026Inject, Align, Recover: Staged Post-Training for Retrieval-Free Document Knowledge Internalizationcited by 0
  37. 2026NeuCME: Toward Dynamic Multimodal Continual Learning via Neural Combinatorics of Multiple Expertscited by 0
  38. 2026Infinite-Parameter LLMs: Generating and Adapting Weights from Live Datacited by 0

50 most cited of 55 All 55 (JSON)