# Prediction Error-based Classification for Class-Incremental Learning

arXiv: https://arxiv.org/abs/2305.18806 · HTML: https://mlsanity.com/paper/2305.18806/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 55 references that are in the map.
- First posted on arXiv 30 May 2023.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [Class-incremental learning: survey and performance evaluation on image classification](https://mlsanity.com/paper/2010.15277/) (2020)
- [Online Continual Learning with Maximally Interfered Retrieval](https://mlsanity.com/paper/1908.04742/) (2019)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [Task-Free Continual Learning](https://mlsanity.com/paper/1812.03596/) (2018)
- [New Insights on Reducing Abrupt Representation Change in Online Continual Learning](https://mlsanity.com/paper/2104.05025/) (2021)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes](https://mlsanity.com/paper/1910.12478/) (2019)
- [Generalization in Deep Learning](https://mlsanity.com/paper/1710.05468/) (2017)
- [Class-Incremental Continual Learning into the eXtended DER-verse](https://mlsanity.com/paper/2201.00766/) (2022)
- [Continuous Learning in Single-Incremental-Task Scenarios](https://mlsanity.com/paper/1806.08568/) (2018)
- [A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks](https://mlsanity.com/paper/2011.01844/) (2020)
- [Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis](https://mlsanity.com/paper/1909.01520/) (2019)
- [A Theoretical Study on Solving Continual Learning](https://mlsanity.com/paper/2211.02633/) (2022)

## Built on by (most cited first, 14 of 14)

- [Continual Learning and Catastrophic Forgetting](https://mlsanity.com/paper/2403.05175/) (2024)
- [Continual Face Forgery Detection via Historical Distribution Preserving](https://mlsanity.com/paper/2308.06217/) (2023)
- [Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also How](https://mlsanity.com/paper/2311.04898/) (2023)
- [Random Representations Outperform Online Continually Learned Representations](https://mlsanity.com/paper/2402.08823/) (2024)
- [Online Continual Learning via Logit Adjusted Softmax](https://mlsanity.com/paper/2311.06460/) (2023)
- [Autoencoder-Based Hybrid Replay for Class-Incremental Learning](https://mlsanity.com/paper/2505.05926/) (2025)
- [No Forgetting Learning: Buffer-free Continual Learning Classification](https://mlsanity.com/paper/2503.04638/) (2025)
- [5G-DIL: Domain Incremental Learning with Similarity-Aware Sampling for Dynamic 5G Indoor Localization](https://mlsanity.com/paper/2505.17684/) (2025)
- [Task-Agnostic Federated Continual Learning via Replay-Free Gradient Projection](https://mlsanity.com/paper/2509.21606/) (2025)
- [Task agnostic continual learning with Pairwise layer architecture](https://mlsanity.com/paper/2405.13632/) (2024)
- [Pathological Prior-Guided Multiple Instance Learning For Mitigating Catastrophic Forgetting in Breast Cancer Whole Slide Image Classification](https://mlsanity.com/paper/2503.06056/) (2025)
- [MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound](https://mlsanity.com/paper/2605.11617/) (2026)
- [Shapley Neuron Values for Continual Learning: Which Neurons Matter Most?](https://mlsanity.com/paper/2605.15877/) (2026)
- [Online Versatile Incremental Learning: Towards Class and Domain-Agnostic Adaptation at Any Time](https://mlsanity.com/paper/2609.36442/) (2026)

## Abstract (arXiv)

Class-incremental learning (CIL) is a particularly challenging variant of continual learning, where the goal is to learn to discriminate between all classes presented in an incremental fashion. Existing approaches often suffer from excessive forgetting and imbalance of the scores assigned to classes that have not been seen together during training. In this study, we introduce a novel approach, Prediction Error-based Classification (PEC), which differs from traditional discriminative and generative classification paradigms. PEC computes a class score by measuring the prediction error of a model trained to replicate the outputs of a frozen random neural network on data from that class. The method can be interpreted as approximating a classification rule based on Gaussian Process posterior variance. PEC offers several practical advantages, including sample efficiency, ease of tuning, and effectiveness even when data are presented one class at a time. Our empirical results show that PEC performs strongly in single-pass-through-data CIL, outperforming other rehearsal-free baselines in all cases and rehearsal-based methods with moderate replay buffer size in most cases across multiple benchmarks.