mlsanityData to 14 Oct 2026

The Importance of Robust Features in Mitigating Catastrophic Forgetting

arXiv 2306.17091 ↗First posted 29 Jun 2023cs.LG cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Continual learning (CL) is an approach to address catastrophic forgetting, which refers to forgetting previously learned knowledge by neural networks when trained on new tasks or data distributions. The adversarial robustness has decomposed features into robust and non-robust types and demonstrated that models trained on robust features significantly enhance adversarial robustness. However, no study has been conducted on the efficacy of robust features from the lens of the CL model in mitigating catastrophic forgetting in CL. In this paper, we introduce the CL robust dataset and train four baseline models on both the standard and CL robust datasets. Our results demonstrate that the CL models trained on the CL robust dataset experienced less catastrophic forgetting of the previously learned tasks than when trained on the standard dataset. Our observations highlight the significance of the features provided to the underlying CL models, showing that CL robust features can alleviate catastrophic forgetting.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 36 of its 50 references that are in the map. First posted on arXiv 29 Jun 2023.

Builds on

36 of 50 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2016Learning without Forgettingcited by 2,496
  5. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  6. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  7. 2016Progressive Neural Networkscited by 1,361
  8. 2017Continual Learning with Deep Generative Replaycited by 1,158
  9. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  10. 2018Experience Replay for Continual Learningcited by 1,095
  11. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  12. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
24 more
  1. 2017Lifelong Learning with Dynamically Expandable Networkscited by 694
  2. 2019Three scenarios for continual learningcited by 614
  3. 2019Gradient based sample selection for online continual learningcited by 571
  4. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 371
  5. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 364
  6. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  7. 2019Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgettingcited by 302
  8. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  9. 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
  10. 2018Selective Experience Replay for Lifelong Learningcited by 241
  11. 2020Semantic Drift Compensation for Class-Incremental Learningcited by 241
  12. 2019Compacting, Picking and Growing for Unforgetting Continual Learningcited by 207
  13. 2022Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidationcited by 113
  14. 2020Generative Feature Replay For Class-Incremental Learningcited by 109
  15. 2022Remember the Past: Distilling Datasets into Addressable Memories for Neural Networkscited by 104
  16. 2019Continual Learning via Neural Pruningcited by 101
  17. 2016Less-forgetting Learning in Deep Neural Networkscited by 97
  18. 2021Distilled Replay: Overcoming Forgetting through Synthetic Samplescited by 36
  19. 2018Continual Classification Learning Using Generative Modelscited by 28
  20. 2018Pseudo-Recursal: Solving the Catastrophic Forgetting Problem in Deep Neural Networkscited by 23
  21. 2023Multimodal Data Integration for Oncology in the Era of Deep Neural Networks: A Reviewcited by 20
  22. 2020Routing Networks with Co-training for Continual Learningcited by 14
  23. 2021Exploring Robust Architectures for Deep Artificial Neural Networkscited by 11
  24. 2022Continual Feature Selection: Spurious Features in Continual Learningcited by 7

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 3.

7 papers.

  1. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 125
  2. 2024Privacy Preserving Federated Learning in Medical Imaging with Uncertainty Estimationcited by 9
  3. 2025A Multimodal Multi-Agent Framework for Radiology Report Generationcited by 8
  4. 2024Adversarially Diversified Rehearsal Memory (ADRM): Mitigating Memory Overfitting Challenge in Continual Learningcited by 2
  5. 2025MorphGen: Morphology-Guided Representation Learning for Robust Single-Domain Generalization in Histopathological Cancer Classificationcited by 2
  6. 2025A Multi-Agent System for Complex Reasoning in Radiology Visual Question Answeringcited by 1
  7. 2024Brain-Inspired Continual Learning-Robust Feature Distillation and Re-Consolidation for Class Incremental Learningcited by 0