# No Representation Rules Them All in Category Discovery

arXiv: https://arxiv.org/abs/2311.17055 · HTML: https://mlsanity.com/paper/2311.17055/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Builds on 65 of its 93 references that are in the map.
- First posted on arXiv 28 Nov 2023.

## Builds on (most cited first, 30 of 65)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding](https://mlsanity.com/paper/2104.12763/) (2021)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [DeiT III: Revenge of the ViT](https://mlsanity.com/paper/2204.07118/) (2022)

## Built on by (most cited first, 30 of 61)

- [UniMatch V2: Pushing the Limit of Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2410.10777/) (2024)
- [SelEx: Self-Expertise in Fine-Grained Generalized Category Discovery](https://mlsanity.com/paper/2408.14371/) (2024)
- [Active Generalized Category Discovery](https://mlsanity.com/paper/2403.04272/) (2024)
- [GET: Unlocking the Multi-modal Potential of CLIP for Generalized Category Discovery](https://mlsanity.com/paper/2403.09974/) (2024)
- [HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shifts](https://mlsanity.com/paper/2408.04591/) (2024)
- [Hyperbolic Category Discovery](https://mlsanity.com/paper/2504.06120/) (2025)
- [ProtoGCD: Unified and Unbiased Prototype Learning for Generalized Category Discovery](https://mlsanity.com/paper/2504.03755/) (2025)
- [Happy: A Debiased Learning Framework for Continual Generalized Category Discovery](https://mlsanity.com/paper/2410.06535/) (2024)
- [DebGCD: Debiased Learning with Distribution Guidance for Generalized Category Discovery](https://mlsanity.com/paper/2504.04804/) (2025)
- [Effective Prompt Pool Learning for Continual Category Discovery](https://mlsanity.com/paper/2407.19001/) (2024)
- [Dissecting Generalized Category Discovery: Multiplex Consensus under Self-Deconstruction](https://mlsanity.com/paper/2508.10731/) (2025)
- [MOS: Modeling Object-Scene Associations in Generalized Category Discovery](https://mlsanity.com/paper/2503.12035/) (2025)
- [Flipped Classroom: Aligning Teacher Attention with Student in Generalized Category Discovery](https://mlsanity.com/paper/2409.19659/) (2024)
- [AnomalyNCD: Towards Novel Anomaly Class Discovery in Industrial Scenarios](https://mlsanity.com/paper/2410.14379/) (2024)
- [Organizing Unstructured Image Collections using Natural Language](https://mlsanity.com/paper/2410.05217/) (2024)
- [Joint Out-of-Distribution Filtering and Data Discovery Active Learning](https://mlsanity.com/paper/2503.02491/) (2025)
- [SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery](https://mlsanity.com/paper/2510.18740/) (2025)
- [Generalized Category Discovery via Reciprocal Learning and Class-Wise Distribution Regularization](https://mlsanity.com/paper/2506.02334/) (2025)
- [A separability-based approach to quantifying generalization: which layer is best?](https://mlsanity.com/paper/2405.01524/) (2024)
- [An Empirical Study into Clustering of Unseen Datasets with Self-Supervised Encoders](https://mlsanity.com/paper/2406.02465/) (2024)
- [A Generic Method for Fine-grained Category Discovery in Natural Language Texts](https://mlsanity.com/paper/2406.13103/) (2024)
- [Exclusive Style Removal for Cross Domain Novel Class Discovery](https://mlsanity.com/paper/2406.18140/) (2024)
- [Category Discovery: An Open-World Perspective](https://mlsanity.com/paper/2509.22542/) (2025)
- [Conditional Representation Learning for Customized Tasks](https://mlsanity.com/paper/2510.04564/) (2025)
- [Beyond Known Clusters: Probe New Prototypes for Efficient Generalized Class Discovery](https://mlsanity.com/paper/2404.08995/) (2024)
- [Labeled Data Selection for Category Discovery](https://mlsanity.com/paper/2406.04898/) (2024)
- [Learning Part Knowledge to Facilitate Category Understanding for Fine-Grained Generalized Category Discovery](https://mlsanity.com/paper/2503.16782/) (2025)
- [Open Ad-hoc Categorization with Contextualized Feature Learning](https://mlsanity.com/paper/2512.16202/) (2025)
- [SpectralGCD: Spectral Concept Selection and Cross-modal Representation Learning for Generalized Category Discovery](https://mlsanity.com/paper/2602.17395/) (2026)
- [FASTTRACK: Fast and Accurate Fact Tracing for LLMs](https://mlsanity.com/paper/2404.15157/) (2024)

## Abstract (arXiv)

In this paper we tackle the problem of Generalized Category Discovery (GCD). Specifically, given a dataset with labelled and unlabelled images, the task is to cluster all images in the unlabelled subset, whether or not they belong to the labelled categories. Our first contribution is to recognize that most existing GCD benchmarks only contain labels for a single clustering of the data, making it difficult to ascertain whether models are using the available labels to solve the GCD task, or simply solving an unsupervised clustering problem. As such, we present a synthetic dataset, named 'Clevr-4', for category discovery. Clevr-4 contains four equally valid partitions of the data, i.e based on object shape, texture, color or count. To solve the task, models are required to extrapolate the taxonomy specified by the labelled set, rather than simply latching onto a single natural grouping of the data. We use this dataset to demonstrate the limitations of unsupervised clustering in the GCD setting, showing that even very strong unsupervised models fail on Clevr-4. We further use Clevr-4 to examine the weaknesses of existing GCD algorithms, and propose a new method which addresses these shortcomings, leveraging consistent findings from the representation learning literature to do so. Our simple solution, which is based on 'mean teachers' and termed μGCD, substantially outperforms implemented baselines on Clevr-4. Finally, when we transfer these findings to real data on the challenging Semantic Shift Benchmark (SSB), we find that μGCD outperforms all prior work, setting a new state-of-the-art. For the project webpage, see https://www.robots.ox.ac.uk/~vgg/data/clevr4/