mlsanityData to 14 Oct 2026

Improving Data-aware and Parameter-aware Robustness for Continual Learning

arXiv 2405.17054 ↗First posted 27 May 2024cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

The goal of Continual Learning (CL) task is to continuously learn multiple new tasks sequentially while achieving a balance between the plasticity and stability of new and old knowledge. This paper analyzes that this insufficiency arises from the ineffective handling of outliers, leading to abnormal gradients and unexpected model updates. To address this issue, we enhance the data-aware and parameter-aware robustness of CL, proposing a Robust Continual Learning (RCL) method. From the data perspective, we develop a contrastive loss based on the concepts of uniformity and alignment, forming a feature distribution that is more applicable to outliers. From the parameter perspective, we present a forward strategy for worst-case perturbation and apply robust gradient projection to the parameters. The experimental results on three benchmarks show that the proposed method effectively maintains robustness and achieves new state-of-the-art (SOTA) results. The code is available at: https://github.com/HanxiXiao/RCL

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 41 of its 56 references that are in the map. First posted on arXiv 27 May 2024.

Builds on

41 of 56 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  5. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  6. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  7. 2017Continual Learning with Deep Generative Replaycited by 1,158
  8. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  9. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  10. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  11. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  12. 2019Large Scale Incremental Learningcited by 732
29 more
  1. 2017Lifelong Learning with Dynamically Expandable Networkscited by 694
  2. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 681
  3. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 521
  4. 2017Sharp Minima Can Generalize For Deep Netscited by 466
  5. 2021Gradient Projection Memory for Continual Learningcited by 348
  6. 2018Continual Learning of Context-dependent Processing in Neural Networkscited by 179
  7. 2018Towards Robust Evaluations of Continual Learningcited by 148
  8. 2019Uncertainty-based Continual Learning with Adaptive Regularizationcited by 146
  9. 2020Adversarial Continual Learningcited by 142
  10. 2021An Empirical Investigation of the Role of Pre-training in Lifelong Learningcited by 134
  11. 2019Scalable and Order-robust Continual Learning with Additive Parameter Decompositioncited by 125
  12. 2021Representational Continuity for Unsupervised Continual Learningcited by 95
  13. 2019Hyperspherical Prototype Networkscited by 86
  14. 2022TRGP: Trust Region Gradient Projection for Continual Learningcited by 82
  15. 2021Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learningcited by 72
  16. 2018Spherical Latent Spaces for Stable Variational Autoencoderscited by 67
  17. 2020CPR: Classifier-Projection Regularization for Continual Learningcited by 62
  18. 2021Relating Adversarially Robust Generalization to Flat Minimacited by 57
  19. 2021Self-Supervised Training Enhances Online Continual Learningcited by 54
  20. 2021Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learningcited by 49
  21. 2022Exploring Example Influence in Continual Learningcited by 44
  22. 2019Robustness of Neural Networks: A Probabilistic and Practical Approachcited by 38
  23. 2022Improving Task-free Continual Learning by Distributionally Robust Memory Evolutioncited by 31
  24. 2020Multi-Domain Multi-Task Rehearsal for Lifelong Learningcited by 24
  25. 2020Targeted Forgetting and False Memory Formation in Continual Learners through Adversarial Backdoor Attackscited by 15
  26. 2022Multi-Label Continual Learning using Augmented Graph Convolutional Networkcited by 12
  27. 2021Adversarial Training of Variational Auto-encoders for Continual Zero-shot Learning(A-CZSL)cited by 5
  28. 2023Adversarial Robust Memory-Based Continual Learnercited by 5
  29. 2023Centroid Distance Distillation for Effective Rehearsal in Continual Learningcited by 4

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Learning Time-Series Representations by Hierarchical Uniformity-Tolerance Latent Balancingcited by 1
  2. 2025Enhancing Memory Recall in LLMs with Gauss-Tin: A Hybrid Instructional and Gaussian Replay Approachcited by 0