# Continual Learning Based on OOD Detection and Task Masking

arXiv: https://arxiv.org/abs/2203.09450 · HTML: https://mlsanity.com/paper/2203.09450/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 61 of its 83 references that are in the map.
- First posted on arXiv 17 Mar 2022.

## Builds on (most cited first, 30 of 61)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [Overcoming catastrophic forgetting with hard attention to the task](https://mlsanity.com/paper/1801.01423/) (2018)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Large Batch Training of Convolutional Networks](https://mlsanity.com/paper/1708.03888/) (2017)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances](https://mlsanity.com/paper/2007.08176/) (2020)
- [Expert Gate: Lifelong Learning with a Network of Experts](https://mlsanity.com/paper/1611.06194/) (2016)
- [PathNet: Evolution Channels Gradient Descent in Super Neural Networks](https://mlsanity.com/paper/1701.08734/) (2017)
- [Recent Advances in Open Set Recognition: A Survey](https://mlsanity.com/paper/1811.08581/) (2018)

## Built on by (most cited first, 21 of 21)

- [A Theoretical Study on Solving Continual Learning](https://mlsanity.com/paper/2211.02633/) (2022)
- [A Survey on Out-of-Distribution Detection in NLP](https://mlsanity.com/paper/2305.03236/) (2023)
- [A Multi-Head Model for Continual Learning via Out-of-Distribution Replay](https://mlsanity.com/paper/2208.09734/) (2022)
- [Open-World Continual Learning: Unifying Novelty Detection and Continual Learning](https://mlsanity.com/paper/2304.10038/) (2023)
- [Prediction Error-based Classification for Class-Incremental Learning](https://mlsanity.com/paper/2305.18806/) (2023)
- [Forgetting, Ignorance or Myopia: Revisiting Key Challenges in Online Continual Learning](https://mlsanity.com/paper/2409.19245/) (2024)
- [Continual Evidential Deep Learning for Out-of-Distribution Detection](https://mlsanity.com/paper/2309.02995/) (2023)
- [Continual Learning in the Presence of Repetition](https://mlsanity.com/paper/2405.04101/) (2024)
- [On the Discrimination and Consistency for Exemplar-Free Class Incremental Learning](https://mlsanity.com/paper/2501.15454/) (2025)
- [Statistical Context Detection for Deep Lifelong Reinforcement Learning](https://mlsanity.com/paper/2405.19047/) (2024)
- [SLE-FNO: Single-Layer Extensions for Task-Agnostic Continual Learning in Fourier Neural Operators](https://mlsanity.com/paper/2603.20410/) (2026)
- [Look At Me, No Replay! SurpriseNet: Anomaly Detection Inspired Class Incremental Learning](https://mlsanity.com/paper/2310.20052/) (2023)
- [HAT-CL: A Hard-Attention-to-the-Task PyTorch Library for Continual Learning](https://mlsanity.com/paper/2307.09653/) (2023)
- [Concept Matching: Clustering-based Federated Continual Learning](https://mlsanity.com/paper/2311.06921/) (2023)
- [Remembering Transformer for Continual Learning](https://mlsanity.com/paper/2404.07518/) (2024)
- [Task-Agnostic Federated Continual Learning via Replay-Free Gradient Projection](https://mlsanity.com/paper/2509.21606/) (2025)
- [Confidence-Based Task Prediction in Continual Disease Classification Using Probability Distribution](https://mlsanity.com/paper/2406.01054/) (2024)
- [Similarity-Aware Mixture-of-Experts for Data-Efficient Continual Learning](https://mlsanity.com/paper/2603.23436/) (2026)
- [GR4CIL: Gap-compensated Routing for CLIP-based Class Incremental Learning](https://mlsanity.com/paper/2604.17822/) (2026)
- [Separating Expert Retention from Autonomous Source Inference in Raw-ECG-Replay-Free Continual ECG Deployment](https://mlsanity.com/paper/2607.01674/) (2026)
- [TOOD: Task-Aware Out-of-Distribution Score Calibration for Continual Learners](https://mlsanity.com/paper/2607.29592/) (2026)

## Abstract (arXiv)

Existing continual learning techniques focus on either task incremental learning (TIL) or class incremental learning (CIL) problem, but not both. CIL and TIL differ mainly in that the task-id is provided for each test sample during testing for TIL, but not provided for CIL. Continual learning methods intended for one problem have limitations on the other problem. This paper proposes a novel unified approach based on out-of-distribution (OOD) detection and task masking, called CLOM, to solve both problems. The key novelty is that each task is trained as an OOD detection model rather than a traditional supervised learning model, and a task mask is trained to protect each task to prevent forgetting. Our evaluation shows that CLOM outperforms existing state-of-the-art baselines by large margins. The average TIL/CIL accuracy of CLOM over six experiments is 87.6/67.9% while that of the best baselines is only 82.4/55.0%.