# Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillation

arXiv: https://arxiv.org/abs/2107.03358 · HTML: https://mlsanity.com/paper/2107.03358/

## Key facts

- Cited by 104 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 39 of its 61 references that are in the map.
- First posted on arXiv 7 Jul 2021.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Contrastive Representation Distillation](https://mlsanity.com/paper/1910.10699/) (2019)
- [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017)
- [Born Again Neural Networks](https://mlsanity.com/paper/1805.04770/) (2018)
- [Variational Information Distillation for Knowledge Transfer](https://mlsanity.com/paper/1904.05835/) (2019)
- [Paraphrasing Complex Network: Network Compression via Factor Transfer](https://mlsanity.com/paper/1802.04977/) (2018)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [Learning to Discover Novel Visual Categories via Deep Transfer Clustering](https://mlsanity.com/paper/1908.09884/) (2019)
- [CrossTransformers: spatially-aware few-shot transfer](https://mlsanity.com/paper/2007.11498/) (2020)
- [Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2102.03526/) (2021)
- [Automatically Discovering and Learning New Visual Categories with Ranking Statistics](https://mlsanity.com/paper/2002.05714/) (2020)
- [SEED: Self-supervised Distillation For Visual Representation](https://mlsanity.com/paper/2101.04731/) (2021)
- [Learning to cluster in order to transfer across domains and tasks](https://mlsanity.com/paper/1711.10125/) (2017)
- [Neighborhood Contrastive Learning for Novel Class Discovery](https://mlsanity.com/paper/2106.10731/) (2021)
- [AutoNovel: Automatically Discovering and Learning Novel Visual Categories](https://mlsanity.com/paper/2106.15252/) (2021)

## Built on by (most cited first, 30 of 104)

- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Generalized Category Discovery](https://mlsanity.com/paper/2201.02609/) (2022)
- [A Unified Objective for Novel Class Discovery](https://mlsanity.com/paper/2108.08536/) (2021)
- [Parametric Classification for Generalized Category Discovery: A Baseline Study](https://mlsanity.com/paper/2211.11727/) (2022)
- [PromptCAL: Contrastive Affinity Learning via Auxiliary Prompts for Generalized Novel Category Discovery](https://mlsanity.com/paper/2212.05590/) (2022)
- [Learning Semi-supervised Gaussian Mixture Models for Generalized Category Discovery](https://mlsanity.com/paper/2305.06144/) (2023)
- [No Representation Rules Them All in Category Discovery](https://mlsanity.com/paper/2311.17055/) (2023)
- [XCon: Learning with Experts for Fine-grained Category Discovery](https://mlsanity.com/paper/2208.01898/) (2022)
- [Towards Open-Set Object Detection and Discovery](https://mlsanity.com/paper/2204.05604/) (2022)
- [Novel Class Discovery without Forgetting](https://mlsanity.com/paper/2207.10659/) (2022)
- [Parametric Information Maximization for Generalized Category Discovery](https://mlsanity.com/paper/2212.00334/) (2022)
- [Learn to Categorize or Categorize to Learn? Self-Coding for Generalized Category Discovery](https://mlsanity.com/paper/2310.19776/) (2023)
- [OpenLDN: Learning to Discover Novel Classes for Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2207.02261/) (2022)
- [Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey](https://mlsanity.com/paper/2407.21794/) (2024)
- [SelEx: Self-Expertise in Fine-Grained Generalized Category Discovery](https://mlsanity.com/paper/2408.14371/) (2024)
- [OpenCon: Open-world Contrastive Learning](https://mlsanity.com/paper/2208.02764/) (2022)
- [Grow and Merge: A Unified Framework for Continuous Categories Discovery](https://mlsanity.com/paper/2210.04174/) (2022)
- [Novel Class Discovery in Semantic Segmentation](https://mlsanity.com/paper/2112.01900/) (2021)
- [Federated Generalized Category Discovery](https://mlsanity.com/paper/2305.14107/) (2023)
- [GET: Unlocking the Multi-modal Potential of CLIP for Generalized Category Discovery](https://mlsanity.com/paper/2403.09974/) (2024)
- [Modeling Inter-Class and Intra-Class Constraints in Novel Class Discovery](https://mlsanity.com/paper/2210.03591/) (2022)
- [CiPR: An Efficient Framework with Cross-instance Positive Relations for Generalized Category Discovery](https://mlsanity.com/paper/2304.06928/) (2023)
- [Solving the Catastrophic Forgetting Problem in Generalized Category Discovery](https://mlsanity.com/paper/2501.05272/) (2025)
- [Incremental Generalized Category Discovery](https://mlsanity.com/paper/2304.14310/) (2023)
- [Novel Class Discovery: an Introduction and Key Concepts](https://mlsanity.com/paper/2302.12028/) (2023)
- [Class-relation Knowledge Distillation for Novel Class Discovery](https://mlsanity.com/paper/2307.09158/) (2023)
- [HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shifts](https://mlsanity.com/paper/2408.04591/) (2024)
- [A Graph-Theoretic Framework for Understanding Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2311.03524/) (2023)
- [When and How Does Known Class Help Discover Unknown Ones? Provable Understanding Through Spectral Analysis](https://mlsanity.com/paper/2308.05017/) (2023)
- [Hyperbolic Category Discovery](https://mlsanity.com/paper/2504.06120/) (2025)

## Abstract (arXiv)

In this paper, we tackle the problem of novel visual category discovery, i.e., grouping unlabelled images from new classes into different semantic partitions by leveraging a labelled dataset that contains images from other different but relevant categories. This is a more realistic and challenging setting than conventional semi-supervised learning. We propose a two-branch learning framework for this problem, with one branch focusing on local part-level information and the other branch focusing on overall characteristics. To transfer knowledge from the labelled data to the unlabelled, we propose using dual ranking statistics on both branches to generate pseudo labels for training on the unlabelled data. We further introduce a mutual knowledge distillation method to allow information exchange and encourage agreement between the two branches for discovering new categories, allowing our model to enjoy the benefits of global and local features. We comprehensively evaluate our method on public benchmarks for generic object classification, as well as the more challenging datasets for fine-grained visual recognition, achieving state-of-the-art performance.