# DNA: Denoised Neighborhood Aggregation for Fine-grained Category Discovery

arXiv: https://arxiv.org/abs/2310.10151 · HTML: https://mlsanity.com/paper/2310.10151/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 38 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2104.14548/) (2021)
- [Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks](https://mlsanity.com/paper/1406.6909/) (2014)
- [Neural Prototype Trees for Interpretable Fine-grained Image Recognition](https://mlsanity.com/paper/2012.02046/) (2020)
- [Generalized Category Discovery](https://mlsanity.com/paper/2201.02609/) (2022)
- [CLEAR: Contrastive Learning for Sentence Representation](https://mlsanity.com/paper/2012.15466/) (2020)
- [Benchmarking Natural Language Understanding Services for building Conversational Agents](https://mlsanity.com/paper/1903.05566/) (2019)
- [Automatically Discovering and Learning New Visual Categories with Ranking Statistics](https://mlsanity.com/paper/2002.05714/) (2020)
- [Fine-Grained Image Analysis with Deep Learning: A Survey](https://mlsanity.com/paper/2111.06119/) (2021)
- [Neighborhood Contrastive Learning for Novel Class Discovery](https://mlsanity.com/paper/2106.10731/) (2021)
- [HDLTex: Hierarchical Deep Learning for Text Classification](https://mlsanity.com/paper/1709.08267/) (2017)
- [Discovering New Intents with Deep Aligned Clustering](https://mlsanity.com/paper/2012.08987/) (2020)
- [Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification](https://mlsanity.com/paper/2109.05427/) (2021)
- [Fine-grained Sentiment Classification using BERT](https://mlsanity.com/paper/1910.03474/) (2019)
- [Generalized Category Discovery with Decoupled Prototypical Network](https://mlsanity.com/paper/2211.15115/) (2022)
- [Fine-grained Angular Contrastive Learning with Coarse Labels](https://mlsanity.com/paper/2012.03515/) (2020)
- [New Intent Discovery with Pre-training and Contrastive Learning](https://mlsanity.com/paper/2205.12914/) (2022)
- [Coarse2Fine: Fine-grained Text Classification on Coarsely-grained Annotated Data](https://mlsanity.com/paper/2109.10856/) (2021)
- [Fine-grained Category Discovery under Coarse-grained supervision with Hierarchical Weighted Self-contrastive Learning](https://mlsanity.com/paper/2210.07733/) (2022)
- [Mitigating Class Boundary Label Uncertainty to Reduce Both Model Bias and Variance](https://mlsanity.com/paper/2002.09963/) (2020)

## Built on by (most cited first, 4 of 4)

- [Generalized Category Discovery with Large Language Models in the Loop](https://mlsanity.com/paper/2312.10897/) (2023)
- [Transfer and Alignment Network for Generalized Category Discovery](https://mlsanity.com/paper/2312.16467/) (2023)
- [A Generic Method for Fine-grained Category Discovery in Natural Language Texts](https://mlsanity.com/paper/2406.13103/) (2024)
- [One Stone, Four Birds: A Comprehensive Solution for QA System Using Supervised Contrastive Learning](https://mlsanity.com/paper/2407.09011/) (2024)

## Abstract (arXiv)

Discovering fine-grained categories from coarsely labeled data is a practical and challenging task, which can bridge the gap between the demand for fine-grained analysis and the high annotation cost. Previous works mainly focus on instance-level discrimination to learn low-level features, but ignore semantic similarities between data, which may prevent these models learning compact cluster representations. In this paper, we propose Denoised Neighborhood Aggregation (DNA), a self-supervised framework that encodes semantic structures of data into the embedding space. Specifically, we retrieve k-nearest neighbors of a query as its positive keys to capture semantic similarities between data and then aggregate information from the neighbors to learn compact cluster representations, which can make fine-grained categories more separatable. However, the retrieved neighbors can be noisy and contain many false-positive keys, which can degrade the quality of learned embeddings. To cope with this challenge, we propose three principles to filter out these false neighbors for better representation learning. Furthermore, we theoretically justify that the learning objective of our framework is equivalent to a clustering loss, which can capture semantic similarities between data to form compact fine-grained clusters. Extensive experiments on three benchmark datasets show that our method can retrieve more accurate neighbors (21.31% accuracy improvement) and outperform state-of-the-art models by a large margin (average 9.96% improvement on three metrics). Our code and data are available at https://github.com/Lackel/DNA.