mlsanityData to 14 Oct 2026

Empirical investigations on WVA structural issues

arXiv 2208.05791 ↗First posted 11 Aug 2022cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper we want to present the results of empirical verification of some issues concerning the methods for overcoming catastrophic forgetting in neural networks. First, in the introduction, we will try to describe in detail the problem of catastrophic forgetting and methods for overcoming it for those who are not yet familiar with this topic. Then we will discuss the essence and limitations of the WVA method which we presented in previous papers. Further, we will touch upon the issues of applying the WVA method to gradients or optimization steps of weights, choosing the optimal attenuation function in this method, as well as choosing the optimal hyper-parameters of the method depending on the number of tasks in sequential training of neural networks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 25 references that are in the map. First posted on arXiv 11 Aug 2022.

Builds on

15 of 25 references.

References · 15

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  3. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  4. 2018Progress & Compress: A scalable framework for continual learningcited by 520
  5. 2013Revisiting Natural Gradient for Deep Networkscited by 159
  6. 2018Differentiable plasticity: training plastic neural networks with backpropagationcited by 84
  7. 2019Hyperbolic Discounting and Learning over Multiple Horizonscited by 60
  8. 2017On Quadratic Penalties in Elastic Weight Consolidationcited by 55
  9. 2019Understanding Generalization through Visualizationscited by 54
  10. 2021Addressing catastrophic forgetting for medical domain expansioncited by 12
  11. 2019Towards continuous learning for glioma segmentation with elastic weight consolidationcited by 11
  12. 2020Enabling Continual Learning with Differentiable Hebbian Plasticitycited by 8
3 more
  1. 2020Sequential Domain Adaptation through Elastic Weight Consolidation for Sentiment Analysiscited by 8
  2. 2021Stabilizing Elastic Weight Consolidation method in practical ML tasks and using weight importances for neural network pruningcited by 8
  3. 2020Natural Way to Overcome the Catastrophic Forgetting in Neural Networkscited by 5

Built on by

Not cited yet.