mlsanityData to 14 Oct 2026

What Must Replay Preserve? Separating Correctable Bias from Class Correspondence

arXiv 2610.07077 ↗First posted 5 Oct 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Class-incremental learning must recognize all classes seen so far without task labels. Logit replay methods such as DER and DER++ mitigate forgetting by matching the model's past predictions on stored examples. Deleting this matching reveals its benefit, but the resulting accuracy cost cannot show whether the stored scores themselves are needed, or whether the cost survives correction of the classifier's bias toward recent classes. We propose a diagnostic framework that treats a cached prediction as temporally heterogeneous supervision: it separates classes known when an example was stored from classes learned afterward, edits each group, and evaluates every model before and after a task-level offset that leaves within-task predictions unchanged. On CIFAR-100 with DER++, suitable fixed constants replace the unrefreshed stored scores of later-learned classes within an equivalence margin of 1 percentage point, and the offset reduces the cost of deleting their matching from 14.9 to 1.8 points. Reassigning the non-gold scores of classes known at storage, which preserves their values and each task's target probability, costs 4.3 points before and 4.0 after the offset, and a parallel cost persists in image distillation. In the tested fixed-head setting, the large cost of deleting later-class matching is thus mostly correctable by this offset, whereas the smaller cost of disrupting class correspondence persists. Code and data are available at anonymous.4open.science/r/replay-preserve-E22B.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 35 references that are in the map. First posted on arXiv 5 Oct 2026.

Builds on

7 of 35 references.

References · 7

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  3. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 911
  4. 2019On Tiny Episodic Memories in Continual Learningcited by 307
  5. 2021New Insights on Reducing Abrupt Representation Change in Online Continual Learningcited by 219
  6. 2022Class-Incremental Continual Learning into the eXtended DER-versecited by 124
  7. 2022A Theoretical Study on Solving Continual Learningcited by 80

Built on by

Not cited yet.