# Grafit: Learning fine-grained image representations with coarse labels

arXiv: https://arxiv.org/abs/2011.12982 · HTML: https://mlsanity.com/paper/2011.12982/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2020.

## Built on by (most cited first, 30 of 43)

- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [DeiT III: Revenge of the ViT](https://mlsanity.com/paper/2204.07118/) (2022)
- [ResNet strikes back: An improved training procedure in timm](https://mlsanity.com/paper/2110.00476/) (2021)
- [ResMLP: Feedforward networks for image classification with data-efficient training](https://mlsanity.com/paper/2105.03404/) (2021)
- [Incorporating Convolution Designs into Visual Transformers](https://mlsanity.com/paper/2103.11816/) (2021)
- [ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias](https://mlsanity.com/paper/2106.03348/) (2021)
- [AutoFormer: Searching Transformers for Visual Recognition](https://mlsanity.com/paper/2107.00651/) (2021)
- [Instance-Conditioned GAN](https://mlsanity.com/paper/2109.05070/) (2021)
- [ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond](https://mlsanity.com/paper/2202.10108/) (2022)
- [Retrieval Augmented Classification for Long-Tail Visual Recognition](https://mlsanity.com/paper/2202.11233/) (2022)
- [MiniViT: Compressing Vision Transformers with Weight Multiplexing](https://mlsanity.com/paper/2204.07154/) (2022)
- [Towards In-context Scene Understanding](https://mlsanity.com/paper/2306.01667/) (2023)
- [GeneCIS: A Benchmark for General Conditional Image Similarity](https://mlsanity.com/paper/2306.07969/) (2023)
- [Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domains](https://mlsanity.com/paper/2201.11528/) (2022)
- [Vocabulary-free Image Classification](https://mlsanity.com/paper/2306.00917/) (2023)
- [Augmenting Convolutional networks with attention-based aggregation](https://mlsanity.com/paper/2112.13692/) (2021)
- [MetaFormer: A Unified Meta Framework for Fine-Grained Recognition](https://mlsanity.com/paper/2203.02751/) (2022)
- [Efficient Algorithms for Learning from Coarse Labels](https://mlsanity.com/paper/2108.09805/) (2021)
- [MaskCon: Masked Contrastive Learning for Coarse-Labelled Dataset](https://mlsanity.com/paper/2303.12756/) (2023)
- [CLIBD: Bridging Vision and Genomics for Biodiversity Monitoring at Scale](https://mlsanity.com/paper/2405.17537/) (2024)
- [Dynamic MLP for Fine-Grained Image Classification by Leveraging Geographical and Temporal Information](https://mlsanity.com/paper/2203.03253/) (2022)
- [On Label Granularity and Object Localization](https://mlsanity.com/paper/2207.10225/) (2022)
- [DenseNets Reloaded: Paradigm Shift Beyond ResNets and ViTs](https://mlsanity.com/paper/2403.19588/) (2024)
- [Semi-Supervised Learning with Taxonomic Labels](https://mlsanity.com/paper/2111.11595/) (2021)
- [Quantifying the Variability Collapse of Neural Networks](https://mlsanity.com/paper/2306.03440/) (2023)
- [It's All in the Head: Representation Knowledge Distillation through Classifier Sharing](https://mlsanity.com/paper/2201.06945/) (2022)
- [Hyperbolic Space with Hierarchical Margin Boosts Fine-Grained Learning from Coarse Labels](https://mlsanity.com/paper/2311.11019/) (2023)
- [Vocabulary-free Image Classification and Semantic Segmentation](https://mlsanity.com/paper/2404.10864/) (2024)
- [Constrained Mean Shift Using Distant Yet Related Neighbors for Representation Learning](https://mlsanity.com/paper/2112.04607/) (2021)

## Abstract (arXiv)

This paper tackles the problem of learning a finer representation than the one provided by training labels. This enables fine-grained category retrieval of images in a collection annotated with coarse labels only. Our network is learned with a nearest-neighbor classifier objective, and an instance loss inspired by self-supervised learning. By jointly leveraging the coarse labels and the underlying fine-grained latent space, it significantly improves the accuracy of category-level retrieval methods. Our strategy outperforms all competing methods for retrieving or classifying images at a finer granularity than that available at train time. It also improves the accuracy for transfer learning tasks to fine-grained datasets, thereby establishing the new state of the art on five public benchmarks, like iNaturalist-2018.