# Novel Class Discovery without Forgetting

arXiv: https://arxiv.org/abs/2207.10659 · HTML: https://mlsanity.com/paper/2207.10659/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 63 references that are in the map.
- First posted on arXiv 21 Jul 2022.

## Builds on (most cited first, 30 of 45)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Self-labelling via simultaneous clustering and representation learning](https://mlsanity.com/paper/1911.05371/) (2019)
- [Variational Information Distillation for Knowledge Transfer](https://mlsanity.com/paper/1904.05835/) (2019)
- [Recent Advances in Open Set Recognition: A Survey](https://mlsanity.com/paper/1811.08581/) (2018)
- [Zero-Shot Learning -- The Good, the Bad and the Ugly](https://mlsanity.com/paper/1703.04394/) (2017)
- [Learning to Discover Novel Visual Categories via Deep Transfer Clustering](https://mlsanity.com/paper/1908.09884/) (2019)
- [Mnemonics Training: Multi-Class Incremental Learning without Forgetting](https://mlsanity.com/paper/2002.10211/) (2020)
- [Generalized Category Discovery](https://mlsanity.com/paper/2201.02609/) (2022)
- [Projected GANs Converge Faster](https://mlsanity.com/paper/2111.01007/) (2021)
- [A Unified Objective for Novel Class Discovery](https://mlsanity.com/paper/2108.08536/) (2021)
- [Adaptive Aggregation Networks for Class-Incremental Learning](https://mlsanity.com/paper/2010.05063/) (2020)
- [Automatically Discovering and Learning New Visual Categories with Ranking Statistics](https://mlsanity.com/paper/2002.05714/) (2020)
- [Learning to cluster in order to transfer across domains and tasks](https://mlsanity.com/paper/1711.10125/) (2017)
- [Learning Placeholders for Open-Set Recognition](https://mlsanity.com/paper/2103.15086/) (2021)

## Built on by (most cited first, 30 of 39)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Learning Semi-supervised Gaussian Mixture Models for Generalized Category Discovery](https://mlsanity.com/paper/2305.06144/) (2023)
- [MetaGCD: Learning to Continually Learn in Generalized Category Discovery](https://mlsanity.com/paper/2308.11063/) (2023)
- [Learn to Categorize or Categorize to Learn? Self-Coding for Generalized Category Discovery](https://mlsanity.com/paper/2310.19776/) (2023)
- [Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey](https://mlsanity.com/paper/2407.21794/) (2024)
- [Federated Generalized Category Discovery](https://mlsanity.com/paper/2305.14107/) (2023)
- [CiPR: An Efficient Framework with Cross-instance Positive Relations for Generalized Category Discovery](https://mlsanity.com/paper/2304.06928/) (2023)
- [Solving the Catastrophic Forgetting Problem in Generalized Category Discovery](https://mlsanity.com/paper/2501.05272/) (2025)
- [Incremental Generalized Category Discovery](https://mlsanity.com/paper/2304.14310/) (2023)
- [Novel Class Discovery: an Introduction and Key Concepts](https://mlsanity.com/paper/2302.12028/) (2023)
- [Class-relation Knowledge Distillation for Novel Class Discovery](https://mlsanity.com/paper/2307.09158/) (2023)
- [HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shifts](https://mlsanity.com/paper/2408.04591/) (2024)
- [Novel Class Discovery for 3D Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2303.11610/) (2023)
- [Hyperbolic Category Discovery](https://mlsanity.com/paper/2504.06120/) (2025)
- [Happy: A Debiased Learning Framework for Continual Generalized Category Discovery](https://mlsanity.com/paper/2410.06535/) (2024)
- [Proxy Anchor-based Unsupervised Learning for Continuous Generalized Category Discovery](https://mlsanity.com/paper/2307.10943/) (2023)
- [DebGCD: Debiased Learning with Distribution Guidance for Generalized Category Discovery](https://mlsanity.com/paper/2504.04804/) (2025)
- [Effective Prompt Pool Learning for Continual Category Discovery](https://mlsanity.com/paper/2407.19001/) (2024)
- [Large-scale Pre-trained Models are Surprisingly Strong in Incremental Novel Class Discovery](https://mlsanity.com/paper/2303.15975/) (2023)
- [Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling](https://mlsanity.com/paper/2403.01053/) (2024)
- [Human-in-the-Loop Visual Re-ID for Population Size Estimation](https://mlsanity.com/paper/2312.05287/) (2023)
- [SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery](https://mlsanity.com/paper/2510.18740/) (2025)
- [Category Adaptation Meets Projected Distillation in Generalized Continual Category Discovery](https://mlsanity.com/paper/2308.12112/) (2023)
- [Exclusive Style Removal for Cross Domain Novel Class Discovery](https://mlsanity.com/paper/2406.18140/) (2024)
- [Online Continuous Generalized Category Discovery](https://mlsanity.com/paper/2408.13492/) (2024)
- [Category Discovery: An Open-World Perspective](https://mlsanity.com/paper/2509.22542/) (2025)
- [Generalized Category Discovery in Semantic Segmentation](https://mlsanity.com/paper/2311.11525/) (2023)
- [Novel class discovery meets foundation models for 3D semantic segmentation](https://mlsanity.com/paper/2312.03782/) (2023)
- [Continual Novel Class Discovery via Feature Enhancement and Adaptation](https://mlsanity.com/paper/2405.06389/) (2024)
- [PartCo: Part-Level Correspondence Priors Enhance Category Discovery](https://mlsanity.com/paper/2509.22769/) (2025)

## Abstract (arXiv)

Humans possess an innate ability to identify and differentiate instances that they are not familiar with, by leveraging and adapting the knowledge that they have acquired so far. Importantly, they achieve this without deteriorating the performance on their earlier learning. Inspired by this, we identify and formulate a new, pragmatic problem setting of NCDwF: Novel Class Discovery without Forgetting, which tasks a machine learning model to incrementally discover novel categories of instances from unlabeled data, while maintaining its performance on the previously seen categories. We propose 1) a method to generate pseudo-latent representations which act as a proxy for (no longer available) labeled data, thereby alleviating forgetting, 2) a mutual-information based regularizer which enhances unsupervised discovery of novel classes, and 3) a simple Known Class Identifier which aids generalized inference when the testing data contains instances form both seen and unseen categories. We introduce experimental protocols based on CIFAR-10, CIFAR-100 and ImageNet-1000 to measure the trade-off between knowledge retention and novel class discovery. Our extensive evaluations reveal that existing models catastrophically forget previously seen categories while identifying novel categories, while our method is able to effectively balance between the competing objectives. We hope our work will attract further research into this newly identified pragmatic problem setting.