mlsanityData to 7 Oct 2026

Continual Feature Selection: Spurious Features in Continual Learning

arXiv 2203.01012 ↗First posted 2 Mar 2022cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Continual Learning (CL) is the research field addressing learning without forgetting when the data distribution is not static. This paper studies spurious features' influence on continual learning algorithms. We show that continual learning algorithms solve tasks by selecting features that are not generalizable. Our experiments highlight that continual learning algorithms face two related problems: (1) spurious features and (2) local spurious features. The first one is due to a covariate shift between training and testing data, while the second is due to the limited access to data at each training step. We study (1) through a consistent set of continual learning experiments varying spurious correlation amount and data distribution support. We show that (2) is a major cause of performance decrease in continual learning along with catastrophic forgetting. This paper presents a different way of understanding performance decrease in continual learning by highlighting the influence of (local) spurious features in algorithms capabilities.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 24 of its 33 references that are in the map. First posted on arXiv 2 Mar 2022.

Builds on

24 of 33 references.

References · 24

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  2. 2019Invariant Risk Minimizationcited by 1,832
  3. 2020In Search of Lost Domain Generalizationcited by 1,042
  4. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  5. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
  6. 2019Orthogonal Gradient Descent for Continual Learningcited by 361
  7. 2019Maintaining Discrimination and Fairness in Class Incremental Learningcited by 281
  8. 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 262
  9. 2018Dataset Distillationcited by 224
  10. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  11. 2018Online Structured Laplace Approximations For Overcoming Catastrophic Forgettingcited by 197
  12. 2019Meta-Learning Representations for Continual Learningcited by 189
12 more
  1. 2020A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrixcited by 99
  2. 2021Continual Learning via Local Module Compositioncited by 61
  3. 2020La-MAML: Look-ahead Meta Learning for Continual Learningcited by 51
  4. 2022Fortuitous Forgetting in Connectionist Networkscited by 39
  5. 2020Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learningcited by 38
  6. 2021Understanding Continual Learning Settings with Data Distribution Drift Analysiscited by 38
  7. 2021Continuum: Simple Management of Complex Continual Learning Scenarioscited by 33
  8. 2021Learning where to learn: Gradient sparsity in meta and continual learningcited by 33
  9. 2019Regularization Shortcomings for Continual Learningcited by 32
  10. 2020Dropout as an Implicit Gating Mechanism For Continual Learningcited by 24
  11. 2020Learning Causal Models Onlinecited by 18
  12. 2021Continual Learning in Deep Networks: an Analysis of the Last Layercited by 17

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 4; 2024: 2.

7 papers.

  1. 2022Beyond Supervised Continual Learning: a Reviewcited by 11
  2. 2023Re-Weighted Softmax Cross-Entropy to Control Forgetting in Federated Learningcited by 9
  3. 2023The Importance of Robust Features in Mitigating Catastrophic Forgettingcited by 7
  4. 2023Principles of Forgetting in Domain-Incremental Semantic Segmentation in Adverse Weather Conditionscited by 4
  5. 2024Watch Your Step: Optimal Retrieval for Continual Learning at Scalecited by 2
  6. 2023Continual Learning in the Presence of Spurious Correlationcited by 1
  7. 2024Selective Attention-based Modulation for Continual Learningcited by 1