mlsanityData to 14 Oct 2026

Mugs: A Multi-Granular Self-Supervised Learning Framework

arXiv 2203.14415 ↗First posted 27 Mar 2022cs.CV cs.AIcited by 54

Not on the map. Why?

Abstract (arXiv)

In self-supervised learning, multi-granular features are heavily desired though rarely investigated, as different downstream tasks (e.g., general and fine-grained classification) often require different or multi-granular features, e.g.~fine- or coarse-grained one or their mixture. In this work, for the first time, we propose an effective MUlti-Granular Self-supervised learning (Mugs) framework to explicitly learn multi-granular visual features. Mugs has three complementary granular supervisions: 1) an instance discrimination supervision (IDS), 2) a novel local-group discrimination supervision (LGDS), and 3) a group discrimination supervision (GDS). IDS distinguishes different instances to learn instance-level fine-grained features. LGDS aggregates features of an image and its neighbors into a local-group feature, and pulls local-group features from different crops of the same image together and push them away for others. It provides complementary instance supervision to IDS via an extra alignment on local neighbors, and scatters different local-groups separately to increase discriminability. Accordingly, it helps learn high-level fine-grained features at a local-group level. Finally, to prevent similar local-groups from being scattered randomly or far away, GDS brings similar samples close and thus pulls similar local-groups together, capturing coarse-grained features at a (semantic) group level. Consequently, Mugs can capture three granular features that often enjoy higher generality on diverse downstream tasks over single-granular features, e.g.~instance-level fine-grained features in contrastive learning. By only pretraining on ImageNet-1K, Mugs sets new SoTA linear probing accuracy 82.1% on ImageNet-1K and improves previous SoTA by 1.1%. It also surpasses SoTAs on other tasks, e.g. transfer learning, detection and segmentation.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Builds on 46 of its 56 references that are in the map. First posted on arXiv 27 Mar 2022.

Builds on

46 of 56 references.

References · 46

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2017Mask R-CNNcited by 7,277
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  10. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  11. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  12. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
34 more
  1. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  2. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  3. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  4. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  5. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  6. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  7. 2017Decoupled Weight Decay Regularizationcited by 1,831
  8. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  9. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  10. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  11. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  12. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  13. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  14. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  15. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  16. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  17. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 903
  18. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  19. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  20. 2018AutoAugment: Learning Augmentation Policies from Datacited by 730
  21. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  22. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 640
  23. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 576
  24. 2021MetaFormer Is Actually What You Need for Visioncited by 524
  25. 2019Cascade R-CNN: High Quality Object Detection and Instance Segmentationcited by 429
  26. 2021PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformerscited by 204
  27. 2021Self-Supervised Learning with Swin Transformerscited by 122
  28. 2021MST: Masked Self-Supervised Transformer for Visual Representationcited by 114
  29. 2019ClusterFit: Improving Generalization of Visual Representationscited by 82
  30. 2019Unsupervised Deep Learning by Neighbourhood Discoverycited by 78
  31. 2016CliqueCNN: Deep Unsupervised Exemplar Learningcited by 41
  32. 2021Prototypical Graph Contrastive Learningcited by 37
  33. 2021DualFormer: Local-Global Stratified Transformer for Efficient Video Recognitioncited by 11
  34. 2021A Theory-Driven Self-Labeling Refinement Method for Contrastive Representation Learningcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 14; 2023: 15; 2024: 15; 2025: 6; 2026: 4.

54 papers.

Sort:

By area:

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  2. 2023A Cookbook of Self-Supervised Learningcited by 250
  3. 2022Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Modelscited by 174
  4. 2022RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rankcited by 153
  5. 2022VICRegL: Self-Supervised Learning of Local Visual Featurescited by 118
  6. 2022On the duality between contrastive and non-contrastive self-supervised learningcited by 108
  7. 2022MILAN: Masked Image Pretraining on Language Assisted Representationcited by 69
  8. 2022LPT: Long-tailed Prompt Tuning for Image Classificationcited by 69
  9. 2023Diffusion Models Beat GANs on Image Classificationcited by 58
  10. 2023Self-supervised learning of Split Invariant Equivariant representationscited by 44
  11. 2023Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Trainingcited by 39
  12. 2023Do text-free diffusion models learn discriminative visual representations?cited by 33
38 more
  1. 2022Guillotine Regularization: Why removing layers is needed to improve generalization in Self-Supervised Learningcited by 26
  2. 2022Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Workscited by 24
  3. 2023Learning high-level visual representations from a child's perspective without strong inductive biasescited by 22
  4. 2025Franca: Nested Matryoshka Clustering for Scalable Visual Representation Learningcited by 21
  5. 2024MIM-Refiner: A Contrastive Learning Boost from Intermediate Pre-Trained Representationscited by 19
  6. 2022Towards Understanding Why Mask-Reconstruction Pretraining Helps in Downstream Taskscited by 18
  7. 2022Attention Distillation: self-supervised vision transformer students need more guidancecited by 16
  8. 2022Neural Eigenfunctions Are Structured Representation Learnerscited by 16
  9. 2024What Variables Affect Out-of-Distribution Generalization in Pretrained Models?cited by 13
  10. 2022SERE: Exploring Feature Self-relation for Self-supervised Transformercited by 12
  11. 2023No More Shortcuts: Realizing the Potential of Temporal Self-Supervisioncited by 11
  12. 2022Towards Sustainable Self-supervised Learningcited by 8
  13. 2024Contrasting Multiple Representations with the Multi-Marginal Matching Gapcited by 8
  14. 2023Visual DNA: Representing and Comparing Images using Distributions of Neuron Activationscited by 7
  15. 2024On Pretraining Data Diversity for Self-Supervised Learningcited by 7
  16. 2024Understanding the Role of Equivariance in Self-supervised Learningcited by 7
  17. 2023Evaluating Self-Supervised Learning via Risk Decompositioncited by 6
  18. 2024In-Context Symmetries: Self-Supervised Learning through Contextual World Modelscited by 5
  19. 2025Exploring Structural Degradation in Dense Representations for Self-supervised Learningcited by 5
  20. 2023Rethinking Evaluation Protocols of Visual Representations Learned via Self-supervised Learningcited by 4
  21. 2024Self-Supervised Representation Learning for Adversarial Attack Detectioncited by 4
  22. 2023FUSC: Fetal Ultrasound Semantic Clustering of Second Trimester Scans Using Deep Self-supervised Learningcited by 3
  23. 2024Multi-Token Enhancing for Vision Representation Learningcited by 3
  24. 2025GONet: A Generalizable Deep Learning Model for Glaucoma Detectioncited by 3
  25. 2025Semantic Concentration for Self-Supervised Dense Representations Learningcited by 3
  26. 2022Spatio-Temporal Crop Aggregation for Video Representation Learningcited by 2
  27. 2023Enhancing Representations through Heterogeneous Self-Supervised Learningcited by 2
  28. 2024Road Surface Friction Estimation for Winter Conditions Utilising General Visual Featurescited by 2
  29. 2023Deep Augmentation: Dropout as Augmentation for Self-Supervised Learningcited by 1
  30. 2023What you see is what you get: Experience ranking with deep neural dataset-to-dataset similarity for topological localisationcited by 1
  31. 2024VideoClusterNet: Self-Supervised and Adaptive Face Clustering For Videoscited by 1
  32. 2024POA: Pre-training Once for Models of All Sizescited by 1
  33. 2025Human Gaze Boosts Object-Centered Representation Learningcited by 1
  34. 2026Implicit Neural Representation Facilitates Unified Universal Vision Encodingcited by 1
  35. 2024GTA: Guided Transfer of Spatial Attention from Object-Centric Representationscited by 0
  36. 2024DatUS^2: Data-driven Unsupervised Semantic Segmentation with Pre-trained Self-supervised Vision Transformercited by 0
  37. 2024Unsqueeze [CLS] Bottleneck to Learn Rich Representationscited by 0
  38. 2024PR-MIM: Delving Deeper into Partial Reconstruction in Masked Image Modelingcited by 0

50 most cited of 54 All 54 (JSON)