# Novel Class Discovery in Semantic Segmentation

arXiv: https://arxiv.org/abs/2112.01900 · HTML: https://mlsanity.com/paper/2112.01900/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 46 references that are in the map.
- First posted on arXiv 3 Dec 2021.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Learning to Adapt Structured Output Space for Semantic Segmentation](https://mlsanity.com/paper/1802.10349/) (2018)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision](https://mlsanity.com/paper/2106.01226/) (2021)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [Unsupervised Domain Adaptation with Residual Transfer Networks](https://mlsanity.com/paper/1602.04433/) (2016)
- [One-Shot Learning for Semantic Segmentation](https://mlsanity.com/paper/1709.03410/) (2017)
- [Prior Guided Feature Enrichment Network for Few-Shot Segmentation](https://mlsanity.com/paper/2008.01449/) (2020)
- [Learning to Discover Novel Visual Categories via Deep Transfer Clustering](https://mlsanity.com/paper/1908.09884/) (2019)
- [Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2003.03773/) (2020)
- [Adaptive Prototype Learning and Allocation for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01893/) (2021)
- [Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision](https://mlsanity.com/paper/2004.07703/) (2020)
- [Hypercorrelation Squeeze for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01538/) (2021)
- [Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals](https://mlsanity.com/paper/2102.06191/) (2021)
- [Feature Weighting and Boosting for Few-Shot Segmentation](https://mlsanity.com/paper/1909.13140/) (2019)
- [PseudoSeg: Designing Pseudo Labels for Semantic Segmentation](https://mlsanity.com/paper/2010.09713/) (2020)
- [PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering](https://mlsanity.com/paper/2103.17070/) (2021)
- [Part-aware Prototype Network for Few-shot Semantic Segmentation](https://mlsanity.com/paper/2007.06309/) (2020)
- [A Unified Objective for Novel Class Discovery](https://mlsanity.com/paper/2108.08536/) (2021)
- [Automatically Discovering and Learning New Visual Categories with Ranking Statistics](https://mlsanity.com/paper/2002.05714/) (2020)

## Built on by (most cited first, 30 of 31)

- [Dynamic Conceptional Contrastive Learning for Generalized Category Discovery](https://mlsanity.com/paper/2303.17393/) (2023)
- [CNOS: A Strong Baseline for CAD-based Novel Object Segmentation](https://mlsanity.com/paper/2307.11067/) (2023)
- [Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentation](https://mlsanity.com/paper/2207.04892/) (2022)
- [Class-incremental Novel Class Discovery](https://mlsanity.com/paper/2207.08605/) (2022)
- [Learn to Categorize or Categorize to Learn? Self-Coding for Generalized Category Discovery](https://mlsanity.com/paper/2310.19776/) (2023)
- [Solving the Catastrophic Forgetting Problem in Generalized Category Discovery](https://mlsanity.com/paper/2501.05272/) (2025)
- [Novel Class Discovery: an Introduction and Key Concepts](https://mlsanity.com/paper/2302.12028/) (2023)
- [Style-Hallucinated Dual Consistency Learning: A Unified Framework for Visual Domain Generalization](https://mlsanity.com/paper/2212.09068/) (2022)
- [Novel Class Discovery for 3D Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2303.11610/) (2023)
- [Happy: A Debiased Learning Framework for Continual Generalized Category Discovery](https://mlsanity.com/paper/2410.06535/) (2024)
- [Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling](https://mlsanity.com/paper/2403.01053/) (2024)
- [Dual-level Adaptive Self-Labeling for Novel Class Discovery in Point Cloud Segmentation](https://mlsanity.com/paper/2407.12489/) (2024)
- [AnomalyNCD: Towards Novel Anomaly Class Discovery in Industrial Scenarios](https://mlsanity.com/paper/2410.14379/) (2024)
- [Category Discovery: An Open-World Perspective](https://mlsanity.com/paper/2509.22542/) (2025)
- [SupeRGB-D: Zero-shot Instance Segmentation in Cluttered Indoor Environments](https://mlsanity.com/paper/2212.11922/) (2022)
- [Generalized Category Discovery in Semantic Segmentation](https://mlsanity.com/paper/2311.11525/) (2023)
- [Novel class discovery meets foundation models for 3D semantic segmentation](https://mlsanity.com/paper/2312.03782/) (2023)
- [PANDAS: Prototype-based Novel Class Discovery and Detection](https://mlsanity.com/paper/2402.17420/) (2024)
- [Open-World Panoptic Segmentation](https://mlsanity.com/paper/2412.12740/) (2024)
- [Generalized Class Discovery in Instance Segmentation](https://mlsanity.com/paper/2502.08149/) (2025)
- [Generalized Fine-Grained Category Discovery with Multi-Granularity Conceptual Experts](https://mlsanity.com/paper/2509.26227/) (2025)
- [Novel Class Discovery for Point Cloud Segmentation via Joint Learning of Causal Representation and Reasoning](https://mlsanity.com/paper/2510.13307/) (2025)
- [Video-based Generalized Category Discovery via Memory-Guided Consistency-Aware Contrastive Learning](https://mlsanity.com/paper/2509.06306/) (2025)
- [Reinforcement Learning Based Multi-modal Feature Fusion Network for Novel Class Discovery](https://mlsanity.com/paper/2308.13801/) (2023)
- [Few-shot Novel Category Discovery](https://mlsanity.com/paper/2505.08260/) (2025)
- [NeurNCD: Novel Class Discovery via Implicit Neural Representation](https://mlsanity.com/paper/2506.06412/) (2025)
- [Intra-view and Inter-view Correlation Guided Multi-view Novel Class Discovery](https://mlsanity.com/paper/2507.12029/) (2025)
- [SIEFormer: Spectral-Interpretable and -Enhanced Transformer for Generalized Category Discovery](https://mlsanity.com/paper/2602.13067/) (2026)
- [Template-based Object Detection Using a Foundation Model](https://mlsanity.com/paper/2603.19773/) (2026)
- [Geometric-Aware Hypergraph Reasoning for Novel Class Discovery in Point Cloud Segmentation](https://mlsanity.com/paper/2606.07280/) (2026)

## Abstract (arXiv)

We introduce a new setting of Novel Class Discovery in Semantic Segmentation (NCDSS), which aims at segmenting unlabeled images containing new classes given prior knowledge from a labeled set of disjoint classes. In contrast to existing approaches that look at novel class discovery in image classification, we focus on the more challenging semantic segmentation. In NCDSS, we need to distinguish the objects and background, and to handle the existence of multiple classes within an image, which increases the difficulty in using the unlabeled data. To tackle this new setting, we leverage the labeled base data and a saliency model to coarsely cluster novel classes for model training in our basic framework. Additionally, we propose the Entropy-based Uncertainty Modeling and Self-training (EUMS) framework to overcome noisy pseudo-labels, further improving the model performance on the novel classes. Our EUMS utilizes an entropy ranking technique and a dynamic reassignment to distill clean labels, thereby making full use of the noisy data via self-supervised learning. We build the NCDSS benchmark on the PASCAL-5i dataset and COCO-20i dataset. Extensive experiments demonstrate the feasibility of the basic framework (achieving an average mIoU of 49.81% on PASCAL-5i) and the effectiveness of EUMS framework (outperforming the basic framework by 9.28% mIoU on PASCAL-5i).