mlsanityData to 14 Oct 2026

Beyond Class Marginals: Bounding Rehearsal Gaps without Freezing Class Co-occurrence

arXiv 2609.25735 ↗First posted 22 Sep 2026cs.LG cs.AI stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Class-balanced replay controls class frequency but does not determine the interval between successive replay appearances of a class. We study this interval, the rehearsal gap, separately from the class marginal and class co-occurrence, and introduce randomised-pass replay (RPR), which visits each resident class once per shuffled pass. For a fixed set of C resident classes and replay batch size b less than or equal to C, RPR preserves the balanced time-averaged class marginal and bounds every gap by 2*ceil(C/b)-1; a churn-conditional bound applies while the resident set changes. The scheduler uses no future class information and adds no replay examples or forward passes. In a linear-head ER-ACE diagnostic, joint absence from the incoming and replay batches produces a one-sided classifier-bias gradient. Longer absence episodes are associated with larger negative bias displacement, and removing the incoming-loss mask attenuates the scheduling effect. In the primary ER-ACE experiments, RPR improves final average accuracy by 0.72-1.67 percentage points relative to independent class-balanced retrieval under reservoir storage, with positive effects also observed under balanced storage. Pretrained ViTs show positive effects on the tested LT10 streams with small replay batches, while matched larger-batch controls show no material effect. Fixed-cycle and reused-pass controls change more than one temporal statistic, so the experiments do not isolate rehearsal-gap length from all other forms of temporal dependence. The accuracy effects depend on the learner and operating regime.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 28 references that are in the map. First posted on arXiv 22 Sep 2026.

Builds on

20 of 28 references.

References · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  4. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  5. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
  6. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 911
  7. 2019Gradient based sample selection for online continual learningcited by 571
  8. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 569
  9. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 379
  10. 2021New Insights on Reducing Abrupt Representation Change in Online Continual Learningcited by 219
  11. 2021Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learningcited by 141
  12. 2020Rethinking Experience Replay: a Bag of Tricks for Continual Learningcited by 94
8 more
  1. 2015Why Random Reshuffling Beats Stochastic Gradient Descentcited by 79
  2. 2018Random Shuffling Beats SGD after Finite Epochscited by 67
  3. 2023Online Prototype Learning for Online Continual Learningcited by 53
  4. 2022Information-theoretic Online Memory Selection for Continual Learningcited by 42
  5. 2021Proximal and Federated Random Reshufflingcited by 33
  6. 2023Improving Online Continual Learning Performance and Stability with Temporal Ensemblescited by 14
  7. 2022Learn the Time to Learn: Replay Scheduling in Continual Learningcited by 11
  8. 2024Watch Your Step: Optimal Retrieval for Continual Learning at Scalecited by 2

Built on by

Not cited yet.