Adaptive Shortcut Debiasing for Online Continual Learning
Not on the map. Why?
Abstract (arXiv)
We propose a novel framework DropTop that suppresses the shortcut bias in online continual learning (OCL) while being adaptive to the varying degree of the shortcut bias incurred by continuously changing environment. By the observed high-attention property of the shortcut bias, highly-activated features are considered candidates for debiasing. More importantly, resolving the limitation of the online environment where prior knowledge and auxiliary data are not ready, two novel techniques -- feature map fusion and adaptive intensity shifting -- enable us to automatically determine the appropriate level and proportion of the candidate shortcut features to be dropped. Extensive experiments on five benchmark datasets demonstrate that, when combined with various OCL algorithms, DropTop increases the average accuracy by up to 10.4% and decreases the forgetting by up to 63.2%.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 31 of its 45 references that are in the map. First posted on arXiv 14 Dec 2023.
Builds on
31 of 45 references.
References · 31
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2020Shortcut Learning in Deep Neural Networkscited by 1,969
- 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
- 2019Natural Adversarial Examplescited by 1,408
- 2018Experience Replay for Continual Learningcited by 1,095
- 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
- 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 911
- 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 742
19 more
- 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 642
- 2019Gradient based sample selection for online continual learningcited by 571
- 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 351
- 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 343
- 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
- 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
- 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 267
- 2019Learning De-biased Representations with Biased Representationscited by 199
- 2018Deep learning generalizes because the parameter-function map is biased towards simple functionscited by 183
- 2021EnD: Entangling and Disentangling deep representations for bias correctioncited by 105
- 2021Exploiting Domain-Specific Features to Enhance Domain Generalizationcited by 86
- 2021Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Modelscited by 78
- 2021Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inferencecited by 67
- 2019SRM : A Style-based Recalibration Module for Convolutional Neural Networkscited by 66
- 2021Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspectivecited by 57
- 2021Shallow Feature Matters for Weakly Supervised Object Localizationcited by 34
- 2021Removing Undesirable Feature Contributions Using Out-of-Distribution Datacited by 23
- 2021Online Continual Adaptation with Active Self-Trainingcited by 8
- 2022FPT: Improving Prompt Tuning Efficiency via Progressive Trainingcited by 3
Built on by
Citing papers per year: 2024: 1; 2025: 0; 2026: 1.
2 papers.