# DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learning

arXiv: https://arxiv.org/abs/2403.17503 · HTML: https://mlsanity.com/paper/2403.17503/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 19 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Builds on (most cited first, 7 of 7)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [Mnemonics Training: Multi-Class Incremental Learning without Forgetting](https://mlsanity.com/paper/2002.10211/) (2020)
- [Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting](https://mlsanity.com/paper/1802.02950/) (2018)
- [Less-forgetting Learning in Deep Neural Networks](https://mlsanity.com/paper/1607.00122/) (2016)

## Built on by (most cited first, 28 of 28)

- [Wave-Mamba: Wavelet State Space Model for Ultra-High-Definition Low-Light Image Enhancement](https://mlsanity.com/paper/2408.01276/) (2024)
- [Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learning](https://mlsanity.com/paper/2409.18265/) (2024)
- [Mixture of Noise for Pre-Trained Model-Based Class-Incremental Learning](https://mlsanity.com/paper/2509.16738/) (2025)
- [AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Models](https://mlsanity.com/paper/2405.16240/) (2024)
- [AIR: Analytic Imbalance Rectifier for Continual Learning](https://mlsanity.com/paper/2408.10349/) (2024)
- [Online Analytic Exemplar-Free Continual Learning with Large Models for Imbalanced Autonomous Driving Task](https://mlsanity.com/paper/2405.17779/) (2024)
- [Towards Efficient and Exact Forgetting Services in Pre-Trained-Model-based Continual Learning](https://mlsanity.com/paper/2505.12239/) (2025)
- [OpenEarthSensing: Large-Scale Fine-Grained Benchmark for Open-World Remote Sensing](https://mlsanity.com/paper/2502.20668/) (2025)
- [Analytic Subspace Routing: How Recursive Least Squares Works in Continual Learning of Large Language Model](https://mlsanity.com/paper/2503.13575/) (2025)
- [Holistic Continual Learning under Concept Drift with Adaptive Memory Realignment](https://mlsanity.com/paper/2507.02310/) (2025)
- [TS-ACL: Closed-Form Solution for Time Series-oriented Continual Learning](https://mlsanity.com/paper/2410.15954/) (2024)
- [OW-CLIP: Data-Efficient Visual Supervision for Open-World Object Detection via Human-AI Collaboration](https://mlsanity.com/paper/2507.19870/) (2025)
- [Forward-Only Continual Learning](https://mlsanity.com/paper/2509.01533/) (2025)
- [CIFNet: An Analytic Neural Learning Framework for Efficient and Calibrated Class-Incremental Learning](https://mlsanity.com/paper/2509.11285/) (2025)
- [AL-GNN: Privacy-Preserving and Replay-Free Continual Graph Learning via Analytic Learning](https://mlsanity.com/paper/2512.18295/) (2025)
- [Analytic Drift Resister for Non-Exemplar Continual Graph Learning](https://mlsanity.com/paper/2604.02633/) (2026)
- [Cross-Sample Relational Fusion: Unifying Domain Generalization and Class-Incremental Learning](https://mlsanity.com/paper/2605.08839/) (2026)
- [Continual learning via probabilistic exchangeable sequence modelling](https://mlsanity.com/paper/2503.20725/) (2025)
- [Gradient-Free Continual Learning](https://mlsanity.com/paper/2504.01219/) (2025)
- [Audio-Visual Class-Incremental Learning for Fish Feeding intensity Assessment in Aquaculture](https://mlsanity.com/paper/2504.15171/) (2025)
- [MCIGLE: Multimodal Exemplar-Free Class-Incremental Graph Learning](https://mlsanity.com/paper/2509.06219/) (2025)
- [Closing the Oracle Gap: Increment Vector Transformation for Class Incremental Learning](https://mlsanity.com/paper/2509.21898/) (2025)
- [Scalable Class-Incremental Learning Based on Parametric Neural Collapse](https://mlsanity.com/paper/2512.21845/) (2025)
- [Analytic Incremental Learning For Sound Source Localization With Imbalance Rectification](https://mlsanity.com/paper/2601.18335/) (2026)
- [Non-Forgetting Knowledge Allocation with Bi-level Competition for Class-Incremental Learning](https://mlsanity.com/paper/2605.29592/) (2026)
- [Geometry-Anchored Transport Framework for Exemplar-Free Class-Incremental Learning](https://mlsanity.com/paper/2606.25347/) (2026)
- [Data-Free Reservoir Features for Efficient Long-Horizon Cold-Start Continual Learning](https://mlsanity.com/paper/2606.27095/) (2026)
- [SPARCL: Spectral Partitioned Analytic Continual Learning](https://mlsanity.com/paper/2608.21307/) (2026)

## Abstract (arXiv)

Class-incremental learning (CIL) under an exemplar-free constraint has presented a significant challenge. Existing methods adhering to this constraint are prone to catastrophic forgetting, far more so than replay-based techniques that retain access to past samples. In this paper, to solve the exemplar-free CIL problem, we propose a Dual-Stream Analytic Learning (DS-AL) approach. The DS-AL contains a main stream offering an analytical (i.e., closed-form) linear solution, and a compensation stream improving the inherent under-fitting limitation due to adopting linear mapping. The main stream redefines the CIL problem into a Concatenated Recursive Least Squares (C-RLS) task, allowing an equivalence between the CIL and its joint-learning counterpart. The compensation stream is governed by a Dual-Activation Compensation (DAC) module. This module re-activates the embedding with a different activation function from the main stream one, and seeks fitting compensation by projecting the embedding to the null space of the main stream's linear mapping. Empirical results demonstrate that the DS-AL, despite being an exemplar-free technique, delivers performance comparable with or better than that of replay-based methods across various datasets, including CIFAR-100, ImageNet-100 and ImageNet-Full. Additionally, the C-RLS' equivalent property allows the DS-AL to execute CIL in a phase-invariant manner. This is evidenced by a never-before-seen 500-phase CIL ImageNet task, which performs on a level identical to a 5-phase one. Our codes are available at https://github.com/ZHUANGHP/Analytic-continual-learning.