# F-OAL: Forward-only Online Analytic Learning with Fast Training and Low Memory Footprint in Class Incremental Learning

arXiv: https://arxiv.org/abs/2403.15751 · HTML: https://mlsanity.com/paper/2403.15751/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 48 references that are in the map.
- First posted on arXiv 23 Mar 2024.

## Builds on (most cited first, 30 of 33)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019)
- [Online Continual Learning with Maximally Interfered Retrieval](https://mlsanity.com/paper/1908.04742/) (2019)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [Online Continual Learning in Image Classification: An Empirical Survey](https://mlsanity.com/paper/2101.10423/) (2021)
- [FOSTER: Feature Boosting and Compression for Class-Incremental Learning](https://mlsanity.com/paper/2204.04662/) (2022)
- [SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model](https://mlsanity.com/paper/2303.05118/) (2023)
- [Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning](https://mlsanity.com/paper/2103.13885/) (2021)
- [Memory Efficient Experience Replay for Streaming Learning](https://mlsanity.com/paper/1809.05922/) (2018)
- [A Unified Continual Learning Framework with General Parameter-Efficient Tuning](https://mlsanity.com/paper/2303.10070/) (2023)
- [FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learning](https://mlsanity.com/paper/2211.13131/) (2022)
- [Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning](https://mlsanity.com/paper/2203.06359/) (2022)
- [Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning](https://mlsanity.com/paper/2403.12030/) (2024)
- [Incremental Learning In Online Scenario](https://mlsanity.com/paper/2003.13191/) (2020)
- [PILOT: A Pre-Trained Model-Based Continual Learning Toolbox](https://mlsanity.com/paper/2309.07117/) (2023)
- [R-DFCIL: Relation-Guided Representation Learning for Data-Free Class Incremental Learning](https://mlsanity.com/paper/2203.13104/) (2022)
- [ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protection](https://mlsanity.com/paper/2205.14922/) (2022)
- [Online Prototype Learning for Online Continual Learning](https://mlsanity.com/paper/2308.00301/) (2023)
- [PCR: Proxy-based Contrastive Replay for Online Class-Incremental Continual Learning](https://mlsanity.com/paper/2304.04408/) (2023)
- [Decoupled Parallel Backpropagation with Convergence Guarantee](https://mlsanity.com/paper/1804.10574/) (2018)
- [Divide and not forget: Ensemble of selectively trained experts in Continual Learning](https://mlsanity.com/paper/2401.10191/) (2024)
- [A simple but strong baseline for online continual learning: Repeated Augmented Rehearsal](https://mlsanity.com/paper/2209.13917/) (2022)
- [DiffClass: Diffusion-Based Class Incremental Learning](https://mlsanity.com/paper/2403.05016/) (2024)
- [Online Continual Learning with Contrastive Vision Transformer](https://mlsanity.com/paper/2207.13516/) (2022)

## Built on by (most cited first, 12 of 12)

- [AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Models](https://mlsanity.com/paper/2405.16240/) (2024)
- [Fly-CL: A Fly-Inspired Framework for Enhancing Efficient Decorrelation and Reduced Training Time in Pre-trained Model-based Continual Representation Learning](https://mlsanity.com/paper/2510.16877/) (2025)
- [REP: Resource-Efficient Prompting for Rehearsal-Free Continual Learning](https://mlsanity.com/paper/2406.04772/) (2024)
- [Analytic Subspace Routing: How Recursive Least Squares Works in Continual Learning of Large Language Model](https://mlsanity.com/paper/2503.13575/) (2025)
- [PROL : Rehearsal Free Continual Learning in Streaming Data via Prompt Online Learning](https://mlsanity.com/paper/2507.12305/) (2025)
- [LibContinual: A Comprehensive Library towards Realistic Continual Learning](https://mlsanity.com/paper/2512.22029/) (2025)
- [DeepAFL: Deep Analytic Federated Learning](https://mlsanity.com/paper/2603.00579/) (2026)
- [Analytic Incremental Learning For Sound Source Localization With Imbalance Rectification](https://mlsanity.com/paper/2601.18335/) (2026)
- [Unlocking Compositional Generalization in Continual Few-Shot Learning](https://mlsanity.com/paper/2605.11710/) (2026)
- [Non-Forgetting Knowledge Allocation with Bi-level Competition for Class-Incremental Learning](https://mlsanity.com/paper/2605.29592/) (2026)
- [Multi-Scale Structural Features for Continual, Comprehensible Visual Recognition in a Developmental Learning Framework](https://mlsanity.com/paper/2607.25531/) (2026)
- [SPARCL: Spectral Partitioned Analytic Continual Learning](https://mlsanity.com/paper/2608.21307/) (2026)

## Abstract (arXiv)

Online Class Incremental Learning (OCIL) aims to train models incrementally, where data arrive in mini-batches, and previous data are not accessible. A major challenge in OCIL is Catastrophic Forgetting, i.e., the loss of previously learned knowledge. Among existing baselines, replay-based methods show competitive results but requires extra memory for storing exemplars, while exemplar-free (i.e., data need not be stored for replay in production) methods are resource-friendly but often lack accuracy. In this paper, we propose an exemplar-free approach--Forward-only Online Analytic Learning (F-OAL). Unlike traditional methods, F-OAL does not rely on back-propagation and is forward-only, significantly reducing memory usage and computational time. Cooperating with a pre-trained frozen encoder with Feature Fusion, F-OAL only needs to update a linear classifier by recursive least square. This approach simultaneously achieves high accuracy and low resource consumption. Extensive experiments on benchmark datasets demonstrate F-OAL's robust performance in OCIL scenarios. Code is available at https://github.com/liuyuchen-cz/F-OAL.