# Error Sensitivity Modulation based Experience Replay: Mitigating Abrupt Representation Drift in Continual Learning

arXiv: https://arxiv.org/abs/2302.11344 · HTML: https://mlsanity.com/paper/2302.11344/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 36 references that are in the map.
- First posted on arXiv 14 Feb 2023.

## Builds on (most cited first, 13 of 13)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning](https://mlsanity.com/paper/2204.04799/) (2022)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [New Insights on Reducing Abrupt Representation Change in Online Continual Learning](https://mlsanity.com/paper/2104.05025/) (2021)
- [Understanding the Role of Training Regimes in Continual Learning](https://mlsanity.com/paper/2006.06958/) (2020)
- [Towards Robust Evaluations of Continual Learning](https://mlsanity.com/paper/1805.09733/) (2018)
- [Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System](https://mlsanity.com/paper/2201.12604/) (2022)
- [Replay in Deep Learning: Current Approaches and Missing Biological Elements](https://mlsanity.com/paper/2104.04132/) (2021)
- [Biologically inspired sleep algorithm for artificial neural networks](https://mlsanity.com/paper/1908.02240/) (2019)

## Built on by (most cited first, 27 of 27)

- [Continual Learning of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2404.16789/) (2024)
- [Class-Incremental Learning with CLIP: Adaptive Representation Adjustment and Parameter Fusion](https://mlsanity.com/paper/2407.14143/) (2024)
- [A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm](https://mlsanity.com/paper/2310.12244/) (2023)
- [Why Fine-Tuning Encourages Hallucinations and How to Fix It](https://mlsanity.com/paper/2604.15574/) (2026)
- [Continual Learning: Forget-free Winning Subnetworks for Video Representations](https://mlsanity.com/paper/2312.11973/) (2023)
- [Approximate Bayesian Class-Conditional Models under Continuous Representation Shift](https://mlsanity.com/paper/2305.19076/) (2023)
- [Adversarial Robust Memory-Based Continual Learner](https://mlsanity.com/paper/2311.17608/) (2023)
- [Progressive Fourier Neural Representation for Sequential Video Compilation](https://mlsanity.com/paper/2306.11305/) (2023)
- [Reconstruct before Query: Continual Missing Modality Learning with Decomposed Prompt Collaboration](https://mlsanity.com/paper/2403.11373/) (2024)
- [Diffusion Model Meets Non-Exemplar Class-Incremental Learning and Beyond](https://mlsanity.com/paper/2408.02983/) (2024)
- [Reducing Class-wise Confusion for Incremental Learning with Disentangled Manifolds](https://mlsanity.com/paper/2503.17677/) (2025)
- [Self-Regulated Neurogenesis for Online Data-Incremental Learning](https://mlsanity.com/paper/2403.14684/) (2024)
- [Look-Ahead Selective Plasticity for Continual Learning of Visual Tasks](https://mlsanity.com/paper/2311.01617/) (2023)
- [Hyperparameter Selection in Continual Learning](https://mlsanity.com/paper/2404.06466/) (2024)
- [Watch Your Step: Optimal Retrieval for Continual Learning at Scale](https://mlsanity.com/paper/2404.10758/) (2024)
- [Mitigating Interference in the Knowledge Continuum through Attention-Guided Incremental Learning](https://mlsanity.com/paper/2405.13978/) (2024)
- [STELLA: Continual Audio-Video Pre-training with Spatio-Temporal Localized Alignment](https://mlsanity.com/paper/2310.08204/) (2023)
- [Tackling Distribution Shift in LLM via KILO: Knowledge-Instructed Learning for Continual Adaptation](https://mlsanity.com/paper/2508.03571/) (2025)
- [Understanding Catastrophic Interference: On the Identifibility of Latent Representations](https://mlsanity.com/paper/2509.23027/) (2025)
- [COLA: Continual Learning via Autoencoder Retrieval of Adapters](https://mlsanity.com/paper/2510.21836/) (2025)
- [Privacy-Aware Continual Self-Supervised Learning on Multi-Window Chest Computed Tomography for Domain-Shift Robustness](https://mlsanity.com/paper/2510.27213/) (2025)
- [Soft-TransFormers for Continual Learning](https://mlsanity.com/paper/2411.16073/) (2024)
- [Unlocking [CLS] Features for Continual Post-Training](https://mlsanity.com/paper/2502.14762/) (2025)
- [Revisiting Deepfake Detection: Chronological Continual Learning and the Limits of Generalization](https://mlsanity.com/paper/2509.07993/) (2025)
- [Global Pre-fixing, Local Adjusting: A Simple yet Effective Contrastive Strategy for Continual Learning](https://mlsanity.com/paper/2509.15347/) (2025)
- [Incremental Human-Object Interaction Detection with Invariant Relation Representation Learning](https://mlsanity.com/paper/2510.27020/) (2025)
- [Stable-Drift: A Patient-Aware Latent Drift Replay Method for Stabilizing Representations in Continual Learning](https://mlsanity.com/paper/2511.22615/) (2025)

## Abstract (arXiv)

Humans excel at lifelong learning, as the brain has evolved to be robust to distribution shifts and noise in our ever-changing environment. Deep neural networks (DNNs), however, exhibit catastrophic forgetting and the learned representations drift drastically as they encounter a new task. This alludes to a different error-based learning mechanism in the brain. Unlike DNNs, where learning scales linearly with the magnitude of the error, the sensitivity to errors in the brain decreases as a function of their magnitude. To this end, we propose textitESMER which employs a principled mechanism to modulate error sensitivity in a dual-memory rehearsal-based system. Concretely, it maintains a memory of past errors and uses it to modify the learning dynamics so that the model learns more from small consistent errors compared to large sudden errors. We also propose textitError-Sensitive Reservoir Sampling to maintain episodic memory, which leverages the error history to pre-select low-loss samples as candidates for the buffer, which are better suited for retaining information. Empirical results show that ESMER effectively reduces forgetting and abrupt drift in representations at the task boundary by gradually adapting to the new task while consolidating knowledge. Remarkably, it also enables the model to learn under high levels of label noise, which is ubiquitous in real-world data streams.