# Data-Free Reservoir Features for Efficient Long-Horizon Cold-Start Continual Learning

arXiv: https://arxiv.org/abs/2606.27095 · HTML: https://mlsanity.com/paper/2606.27095/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 44 references that are in the map.
- First posted on arXiv 25 Jun 2026.

## Builds on (most cited first, 20 of 20)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Semantic Drift Compensation for Class-Incremental Learning](https://mlsanity.com/paper/2004.00440/) (2020)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [Generalization Properties of Learning with Random Features](https://mlsanity.com/paper/1602.04474/) (2016)
- [RanPAC: Random Projections and Pre-trained Models for Continual Learning](https://mlsanity.com/paper/2307.02251/) (2023)
- [FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learning](https://mlsanity.com/paper/2211.13131/) (2022)
- [Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis](https://mlsanity.com/paper/1909.01520/) (2019)
- [FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learning](https://mlsanity.com/paper/2309.14062/) (2023)
- [PyCIL: A Python Toolbox for Class-Incremental Learning](https://mlsanity.com/paper/2112.12533/) (2021)
- [Isambard-AI: a leadership class supercomputer optimised specifically for Artificial Intelligence](https://mlsanity.com/paper/2410.11199/) (2024)
- [ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protection](https://mlsanity.com/paper/2205.14922/) (2022)
- [Elastic Feature Consolidation for Cold Start Exemplar-Free Incremental Learning](https://mlsanity.com/paper/2402.03917/) (2024)
- [Resurrecting Old Classes with New Data for Exemplar-Free Continual Learning](https://mlsanity.com/paper/2405.19074/) (2024)
- [Exemplar-free Continual Representation Learning via Learnable Drift Compensation](https://mlsanity.com/paper/2407.08536/) (2024)
- [DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learning](https://mlsanity.com/paper/2403.17503/) (2024)
- [Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learning](https://mlsanity.com/paper/2409.18265/) (2024)
- [Reinforcement Learning with Convolutional Reservoir Computing](https://mlsanity.com/paper/1912.04161/) (2019)
- [Continual Learning of Dynamical Systems with Competitive Federated Reservoir Computing](https://mlsanity.com/paper/2206.13336/) (2022)
- [EFC++: Elastic Feature Consolidation with Prototype Re-balancing for Cold Start Exemplar-free Incremental Learning](https://mlsanity.com/paper/2503.10439/) (2025)

## Abstract (arXiv)

Cold-start exemplar-free class-incremental learning requires learning a growing set of classes without replay, external pretraining, or a large initial task. Existing cold-start methods typically either train the backbone throughout the stream and compensate for semantic drift, or freeze a backbone after the first task, producing features biased toward the initial classes. These choices also create a computational tension: drift-compensation methods require repeated backbone training and increasingly expensive updates as the task horizon grows, while frozen-backbone methods are cheap but weak under cold start. We study a third option: a feature extractor that is never fit to image data at all. We propose CIRCLE, a class-incremental classifier built from fixed bidirectional two-dimensional reservoir features, adapted from BiRC2D for image classification, and streaming linear discriminant analysis heads. CIRCLE groups multiple random reservoir instantiations into feature ensembles and averages the softmax outputs of independent SLDA heads, yielding a tunable bias-variance tradeoff between richer random features and prediction-level ensembling. Because the feature extractor is fixed and the head admits streaming closed-form updates, CIRCLE performs sample-wise training without replay, task-boundary information, or backbone backpropagation. On CIFAR-100, TinyImageNet, ImageNet-Subset, and ImageNet-1k, CIRCLE is competitive at 10-20 task splits and substantially outperforms strong CS-EFCIL baselines at 50, 100, and 500 task splits, while training much faster than trained-backbone drift-compensation methods. Ablations show that the BiRC2D-style extractor, SLDA head, and balanced feature/prediction ensembling each contribute to the final performance.