Cross-Level Multi-Instance Distillation for Self-Supervised Fine-Grained Visual Categorization
Not on the map. Why?
Abstract (arXiv)
High-quality annotation of fine-grained visual categories demands great expert knowledge, which is taxing and time consuming. Alternatively, learning fine-grained visual representation from enormous unlabeled images (e.g., species, brands) by self-supervised learning becomes a feasible solution. However, recent researches find that existing self-supervised learning methods are less qualified to represent fine-grained categories. The bottleneck lies in that the pre-text representation is built from every patch-wise embedding, while fine-grained categories are only determined by several key patches of an image. In this paper, we propose a Cross-level Multi-instance Distillation (CMD) framework to tackle the challenge. Our key idea is to consider the importance of each image patch in determining the fine-grained pre-text representation by multiple instance learning. To comprehensively learn the relation between informative patches and fine-grained semantics, the multi-instance knowledge distillation is implemented on both the region/image crop pairs from the teacher and student net, and the region-image crops inside the teacher / student net, which we term as intra-level multi-instance distillation and inter-level multi-instance distillation. Extensive experiments on CUB-200-2011, Stanford Cars and FGVC Aircraft show that the proposed method outperforms the contemporary method by upto 10.14% and existing state-of-the-art self-supervised learning approaches by upto 19.78% on both top-1 accuracy and Rank-1 retrieval metric.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 59 of its 83 references that are in the map. First posted on arXiv 16 Jan 2024.
Builds on
59 of 83 references.
References · 59
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2020Exploring Simple Siamese Representation Learningcited by 2,853
- 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
- 2017Deep Setscited by 1,394
- 2018Attention-based Deep Multiple Instance Learningcited by 1,077
- 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
38 more
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
- 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
- 2019Fixing the train-test resolution discrepancycited by 209
- 2021Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encodingcited by 207
- 2021Efficient Self-supervised Vision Transformers for Representation Learningcited by 149
- 2021Fine-Grained Image Analysis with Deep Learning: A Surveycited by 135
- 2021TransFG: A Transformer Architecture for Fine-grained Recognitioncited by 131
- 2016Revisiting Multiple Instance Neural Networkscited by 119
- 2017Multiple Instance Detection Network with Online Instance Classifier Refinementcited by 117
- 2021Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identificationcited by 114
- 2020Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patchescited by 92
- 2021When Does Contrastive Visual Representation Learning Work?cited by 92
- 2020Learning Attentive Pairwise Interaction for Fine-Grained Classificationcited by 84
- 2022Dual Cross-Attention Learning for Fine-Grained Visual Categorization and Object Re-Identificationcited by 78
- 2022Crafting Better Contrastive Views for Siamese Representation Learningcited by 67
- 2019Weakly Supervised Complementary Parts Models for Fine-Grained Image Classification from the Bottom Upcited by 66
- 2022Sparse Instance Activation for Real-Time Instance Segmentationcited by 63
- 2019Cross-X Learning for Fine-Grained Visual Categorizationcited by 61
- 2019Attention Convolutional Binary Neural Tree for Fine-Grained Visual Categorizationcited by 51
- 2020Channel Interaction Networks for Fine-Grained Image Categorizationcited by 47
- 2023Learn to Categorize or Categorize to Learn? Self-Coding for Generalized Category Discoverycited by 39
- 2020Distilling Localization for Self-Supervised Representation Learningcited by 34
- 2021RAMS-Trans: Recurrent Attention Multi-scale Transformer forFine-grained Image Recognitioncited by 34
- 2023Reverse Engineering Self-Supervised Learningcited by 33
- 2021Context-aware Attentional Pooling (CAP) for Fine-grained Visual Classificationcited by 31
- 2022Learning Where to Learn in Cross-View Self-Supervised Learningcited by 31
- 2022Fine-Grained Object Classification via Self-Supervised Pose Alignmentcited by 22
- 2023Semi-supervised learning made simple with self-supervised clusteringcited by 20
- 2022Fast Differentiable Matrix Square Root and Inverse Square Rootcited by 16
- 2023Learning Gabor Texture Features for Fine-Grained Recognitioncited by 16
- 2019Multiple instance dense connected convolution neural network for aerial image scene classificationcited by 15
- 2022Dynamic MLP for Fine-Grained Image Classification by Leveraging Geographical and Temporal Informationcited by 15
- 2023Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problemscited by 14
- 2023Delving into Multimodal Prompting for Fine-grained Visual Classificationcited by 14
- 2022Improving Fine-Grained Visual Recognition in Low Data Regimes via Self-Boosting Attention Mechanismcited by 12
- 2023Coreset Sampling from Open-Set for Fine-Grained Self-Supervised Learningcited by 12
- 2022SEMICON: A Learning-to-hash Solution for Large-scale Fine-grained Image Retrievalcited by 11
- 2022Label-efficient Hybrid-supervised Learning for Medical Image Segmentationcited by 10
50 most cited of 59 All 59 (JSON)
Built on by
Citing papers per year: 2025: 1.
1 paper.