# ViTNF: Leveraging Neural Fields to Boost Vision Transformers in Generalized Category Discovery

arXiv: https://arxiv.org/abs/2506.02367 · HTML: https://mlsanity.com/paper/2506.02367/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 21 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [On the Relationship between Self-Attention and Convolutional Layers](https://mlsanity.com/paper/1911.03584/) (2019)
- [Large-Scale Study of Curiosity-Driven Learning](https://mlsanity.com/paper/1808.04355/) (2018)
- [Generalized Category Discovery](https://mlsanity.com/paper/2201.02609/) (2022)
- [Parametric Classification for Generalized Category Discovery: A Baseline Study](https://mlsanity.com/paper/2211.11727/) (2022)
- [Dynamic Conceptional Contrastive Learning for Generalized Category Discovery](https://mlsanity.com/paper/2303.17393/) (2023)
- [PromptCAL: Contrastive Affinity Learning via Auxiliary Prompts for Generalized Novel Category Discovery](https://mlsanity.com/paper/2212.05590/) (2022)
- [Learning Semi-supervised Gaussian Mixture Models for Generalized Category Discovery](https://mlsanity.com/paper/2305.06144/) (2023)
- [No Representation Rules Them All in Category Discovery](https://mlsanity.com/paper/2311.17055/) (2023)
- [OpenLDN: Learning to Discover Novel Classes for Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2207.02261/) (2022)
- [Active Generalized Category Discovery](https://mlsanity.com/paper/2403.04272/) (2024)
- [A Graph-Theoretic Framework for Understanding Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2311.03524/) (2023)

## Abstract (arXiv)

Generalized category discovery (GCD) is a highly popular task in open-world recognition, aiming to identify unknown class samples using known class data. By leveraging pre-training, meta-training, and fine-tuning, ViT achieves excellent few-shot learning capabilities. Its MLP head is a feedforward network, trained synchronously with the entire network in the same process, increasing the training cost and difficulty without fully leveraging the power of the feature extractor. This paper proposes a new architecture by replacing the MLP head with a neural field-based one. We first present a new static neural field function to describe the activity distribution of the neural field and then use two static neural field functions to build an efficient few-shot classifier. This neural field-based (NF) classifier consists of two coupled static neural fields. It stores the feature information of support samples by its elementary field, the known categories by its high-level field, and the category information of support samples by its cross-field connections. We replace the MLP head with the proposed NF classifier, resulting in a novel architecture ViTNF, and simplify the three-stage training mode by pre-training the feature extractor on source tasks and training the NF classifier with support samples in meta-testing separately, significantly reducing ViT's demand for training samples and the difficulty of model training. To enhance the model's capability in identifying new categories, we provide an effective algorithm to determine the lateral interaction scale of the elementary field. Experimental results demonstrate that our model surpasses existing state-of-the-art methods on CIFAR-100, ImageNet-100, CUB-200, and Standard Cars, achieving dramatic accuracy improvements of 19% and 16% in new and all classes, respectively, indicating a notable advantage in GCD.