# CIFNet: An Analytic Neural Learning Framework for Efficient and Calibrated Class-Incremental Learning

arXiv: https://arxiv.org/abs/2509.11285 · HTML: https://mlsanity.com/paper/2509.11285/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 14 Sep 2025.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning](https://mlsanity.com/paper/2211.13218/) (2022)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [FOSTER: Feature Boosting and Compression for Class-Incremental Learning](https://mlsanity.com/paper/2204.04662/) (2022)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need](https://mlsanity.com/paper/2303.07338/) (2023)
- [SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model](https://mlsanity.com/paper/2303.05118/) (2023)
- [A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learning](https://mlsanity.com/paper/2205.13218/) (2022)
- [Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimality](https://mlsanity.com/paper/2310.07234/) (2023)
- [Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning](https://mlsanity.com/paper/2403.12030/) (2024)
- [ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protection](https://mlsanity.com/paper/2205.14922/) (2022)
- [Mixture of Experts Meets Prompt-Based Continual Learning](https://mlsanity.com/paper/2405.14124/) (2024)
- [DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learning](https://mlsanity.com/paper/2403.17503/) (2024)
- [Adapt Your Teacher: Improving Knowledge Distillation for Exemplar-free Continual Learning](https://mlsanity.com/paper/2308.09544/) (2023)
- [Task-Agnostic Guided Feature Expansion for Class-Incremental Learning](https://mlsanity.com/paper/2503.00823/) (2025)
- [Reducing Class-wise Confusion for Incremental Learning with Disentangled Manifolds](https://mlsanity.com/paper/2503.17677/) (2025)
- [EXACFS -- A CIL Method to mitigate Catastrophic Forgetting](https://mlsanity.com/paper/2410.23751/) (2024)

## Built on by (most cited first, 1 of 1)

- [HydraCIL: Decoupled Class-Incremental Learning through Prototype-Guided Multi-Head Classifiers](https://mlsanity.com/paper/2606.09960/) (2026)

## Abstract (arXiv)

Class-Incremental Learning (CIL) in deep neural networks is conventionally framed as an iterative gradient-based optimization problem, incurring high computational cost, hyperparameter sensitivity, and risk of catastrophic forgetting. In this work, we demonstrate that when leveraging frozen pre-trained representations, CIL can be solved as a sequence of deterministic, closed-form classifier adaptations without backpropagation or iterative convergence. We propose CIFNet, an analytic neural learning framework built upon Regularised Recursive Least-Squares (RRLS). CIFNet updates classifier weights via an exact, closed-form ridge-regression solution operating in a stationary embedding space. To counteract the structural initialisation bias that arises when newly expanded output neurons are introduced without exposure to past-class evidence, CIFNet incorporates a lightweight calibration buffer in latent space alongside density-aware oversampling, ensuring globally balanced decision boundaries without raw image storage or gradient updates. Extensive evaluations across CIFAR-100, ImageNet-100, and CORe50 show that CIFNet achieves predictive accuracy competitive with iterative CIL baselines while maintaining strictly monotonic, smooth learning trajectories free from intermediate performance collapse. Furthermore, by replacing epoch-wise backpropagation with closed-form moment accumulation, CIFNet achieves up to 20× reduction in energy consumption. These findings establish calibrated analytic learning as an efficient, stable, and mathematically grounded paradigm for continual adaptation in neural networks.