mlsanityData to 14 Oct 2026

Behavioral Experiments for Understanding Catastrophic Forgetting

arXiv 2110.10570 ↗First posted 20 Oct 2021cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

In this paper we explore whether the fundamental tool of experimental psychology, the behavioral experiment, has the power to generate insight not only into humans and animals, but artificial systems too. We apply the techniques of experimental psychology to investigating catastrophic forgetting in neural networks. We present a series of controlled experiments with two-layer ReLU networks, and exploratory results revealing a new understanding of the behavior of catastrophic forgetting. Alongside our empirical findings, we demonstrate an alternative, behavior-first approach to investigating neural network phenomena.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 28 of its 60 references that are in the map. First posted on arXiv 20 Oct 2021.

Builds on

28 of 60 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2013Intriguing properties of neural networkscited by 5,155
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  7. 2016Learning without Forgettingcited by 2,496
  8. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  9. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  10. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  11. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  12. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
16 more
  1. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  2. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
  3. 2017Exploring Generalization in Deep Learningcited by 600
  4. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 571
  5. 2020What is being transferred in transfer learning?cited by 368
  6. 2017Measuring Catastrophic Forgetting in Neural Networkscited by 360
  7. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 306
  8. 2021Scaling Laws for Transfercited by 272
  9. 2018Stress Test Evaluation for Natural Language Inferencecited by 213
  10. 2018Online Structured Laplace Approximations For Overcoming Catastrophic Forgettingcited by 201
  11. 2020Understanding the Role of Training Regimes in Continual Learningcited by 180
  12. 2020Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semanticscited by 161
  13. 2018Towards Robust Evaluations of Continual Learningcited by 148
  14. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 106
  15. 2021Continual Learning in the Teacher-Student Setup: Impact of Task Similaritycited by 88
  16. 2021Perspectives on Machine Learning from Psychology's Reproducibility Crisiscited by 9

Built on by

Cited by year

Citing papers per year: 2022: 2.

2 papers.

  1. 2022The Effect of Task Ordering in Continual Learningcited by 31
  2. 2022Maslow's Hammer for Catastrophic Forgetting: Node Re-Use vs Node Activationcited by 16