mlsanityData to 14 Oct 2026

NeuroGuard: Neural Gradient Update Aware of Representation Damage

arXiv 2608.08068 ↗First posted 8 Aug 2026cs.CV cs.LGcited by 0

On the map: Knowledge distillation → Distillation for continual learning

Abstract (arXiv)

Long-tailed class-incremental learning (LT-CIL) must learn new classes from imbalanced streams while retaining old classes. Existing methods mainly change replay, classifiers, or losses. We study a different factor, namely how strongly the feature representation should be updated at each task boundary. We propose NeuroGuard, an update-control method added to DGR, a replay-based LT-CIL baseline, without adding learnable parameters. NeuroGuard preserves DGR's replay memory, classifier, and set of loss terms. Adaptive Gradient Scaling (AGS) converts teacher uncertainty into one task-wise gradient scale. Confidence-Ranked Knowledge Distillation Reweighting (CRK) gives larger knowledge-distillation weights to replay samples that the teacher predicts less decisively. Fragility-Blended Entropy Gate (FBE) adds old-memory leakage to the scale decision. Across five LT-CIL settings, NeuroGuard improves over DGR in every setting. In the four main benchmark comparisons, it achieves the best task-agnostic accuracy among the compared methods. The gains extend to both old- and new-class accuracy, while medium-frequency accuracy improves consistently across all five settings. Controlled comparisons show that the gain does not come from generic gradient suppression: AGS outperforms a matched fixed-scale control in all five settings, demonstrating that boundary-specific scaling is more effective than applying the same average scale throughout learning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Distillation for continual learning, one of 12 directions in Knowledge distillation. Builds on 24 of its 37 references that are in the map. First posted on arXiv 8 Aug 2026.

Builds on

24 of 37 references.

References · 24

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  3. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  4. 2018Experience Replay for Continual Learningcited by 1,095
  5. 2020Long-tail learning via logit adjustmentcited by 618
  6. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  7. 2021Gradient Projection Memory for Continual Learningcited by 348
  8. 2022FOSTER: Feature Boosting and Compression for Class-Incremental Learningcited by 229
  9. 2023Class-Incremental Learning: A Surveycited by 209
  10. 2020A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learningcited by 87
  11. 2020Imbalanced Continual Learning with Partitioning Reservoir Samplingcited by 72
  12. 2023Continual Learning: Applications and the Road Forwardcited by 59
12 more
  1. 2021Co-Transport for Class-Incremental Learningcited by 38
  2. 2024Gradient Reweighting: Towards Imbalanced Class-Incremental Learningcited by 37
  3. 2022Long-Tailed Class Incremental Learningcited by 36
  4. 2021Synaptic metaplasticity in binarized neural networkscited by 29
  5. 2020Prime-Aware Adaptive Distillationcited by 28
  6. 2023Long-Tailed Continual Learning For Visual Food Recognitioncited by 20
  7. 2023MDCS: More Diverse Experts with Consistency Self-distillation for Long-tailed Recognitioncited by 19
  8. 2022Rebalancing Batch Normalization for Exemplar-based Class-Incremental Learningcited by 16
  9. 2023Long-Tailed Recognition by Mutual Information Maximization between Latent Features and Ground-Truth Labelscited by 16
  10. 2023Subspace Distillation for Continual Learningcited by 12
  11. 2024Future-Proofing Class-Incremental Learningcited by 4
  12. 2021Principal Gradient Direction and Confidence Reservoir Sampling for Continual Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidation
  2. 2024Reducing Catastrophic Forgetting in Online Class Incremental Learning Using Self-Distillation
  3. 2023Fine-Grained Knowledge Selection and Restoration for Non-Exemplar Class Incremental Learning
  4. 2023SRIL: Selective Regularization for Class-Incremental Learning
  5. 2023New Insights on Relieving Task-Recency Bias for Online Class Incremental Learning
  6. 2021Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning
  7. 2026LoGRA: Scaling LLM Reinforcement Learning with Low-Rank Gradient Sketches
  8. 2022Few-Shot Class-Incremental Learning via Entropy-Regularized Data-Free Replay