# NeuroGuard: Neural Gradient Update Aware of Representation Damage

arXiv: https://arxiv.org/abs/2608.08068 · HTML: https://mlsanity.com/paper/2608.08068/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Distillation for continual learning, one of 12 directions in Knowledge distillation.
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 8 Aug 2026.

## Placed in

- Knowledge distillation / [Distillation for continual learning](https://mlsanity.com/distill/distillation-for-continual-learning/)

## Builds on (most cited first, 24 of 24)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [DER: Dynamically Expandable Representation for Class Incremental Learning](https://mlsanity.com/paper/2103.16788/) (2021)
- [Gradient Projection Memory for Continual Learning](https://mlsanity.com/paper/2103.09762/) (2021)
- [FOSTER: Feature Boosting and Compression for Class-Incremental Learning](https://mlsanity.com/paper/2204.04662/) (2022)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning](https://mlsanity.com/paper/2009.01797/) (2020)
- [Imbalanced Continual Learning with Partitioning Reservoir Sampling](https://mlsanity.com/paper/2009.03632/) (2020)
- [Continual Learning: Applications and the Road Forward](https://mlsanity.com/paper/2311.11908/) (2023)
- [Co-Transport for Class-Incremental Learning](https://mlsanity.com/paper/2107.12654/) (2021)
- [Gradient Reweighting: Towards Imbalanced Class-Incremental Learning](https://mlsanity.com/paper/2402.18528/) (2024)
- [Long-Tailed Class Incremental Learning](https://mlsanity.com/paper/2210.00266/) (2022)
- [Synaptic metaplasticity in binarized neural networks](https://mlsanity.com/paper/2101.07592/) (2021)
- [Prime-Aware Adaptive Distillation](https://mlsanity.com/paper/2008.01458/) (2020)
- [Long-Tailed Continual Learning For Visual Food Recognition](https://mlsanity.com/paper/2307.00183/) (2023)
- [MDCS: More Diverse Experts with Consistency Self-distillation for Long-tailed Recognition](https://mlsanity.com/paper/2308.09922/) (2023)
- [Rebalancing Batch Normalization for Exemplar-based Class-Incremental Learning](https://mlsanity.com/paper/2201.12559/) (2022)
- [Long-Tailed Recognition by Mutual Information Maximization between Latent Features and Ground-Truth Labels](https://mlsanity.com/paper/2305.01160/) (2023)
- [Subspace Distillation for Continual Learning](https://mlsanity.com/paper/2307.16419/) (2023)
- [Future-Proofing Class-Incremental Learning](https://mlsanity.com/paper/2404.03200/) (2024)
- [Principal Gradient Direction and Confidence Reservoir Sampling for Continual Learning](https://mlsanity.com/paper/2108.09592/) (2021)

## Abstract (arXiv)

Long-tailed class-incremental learning (LT-CIL) must learn new classes from imbalanced streams while retaining old classes. Existing methods mainly change replay, classifiers, or losses. We study a different factor, namely how strongly the feature representation should be updated at each task boundary. We propose NeuroGuard, an update-control method added to DGR, a replay-based LT-CIL baseline, without adding learnable parameters. NeuroGuard preserves DGR's replay memory, classifier, and set of loss terms. Adaptive Gradient Scaling (AGS) converts teacher uncertainty into one task-wise gradient scale. Confidence-Ranked Knowledge Distillation Reweighting (CRK) gives larger knowledge-distillation weights to replay samples that the teacher predicts less decisively. Fragility-Blended Entropy Gate (FBE) adds old-memory leakage to the scale decision. Across five LT-CIL settings, NeuroGuard improves over DGR in every setting. In the four main benchmark comparisons, it achieves the best task-agnostic accuracy among the compared methods. The gains extend to both old- and new-class accuracy, while medium-frequency accuracy improves consistently across all five settings. Controlled comparisons show that the gain does not come from generic gradient suppression: AGS outperforms a matched fixed-scale control in all five settings, demonstrating that boundary-specific scaling is more effective than applying the same average scale throughout learning.