mlsanityData to 7 Oct 2026

HIRL: A General Framework for Hierarchical Image Representation Learning

arXiv 2205.13159 ↗First posted 26 May 2022cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Learning self-supervised image representations has been broadly studied to boost various visual understanding tasks. Existing methods typically learn a single level of image semantics like pairwise semantic similarity or image clustering patterns. However, these methods can hardly capture multiple levels of semantic information that naturally exists in an image dataset, e.g., the semantic hierarchy of "Persian cat to cat to mammal" encoded in an image database for species. It is thus unknown whether an arbitrary image self-supervised learning (SSL) approach can benefit from learning such hierarchical semantics. To answer this question, we propose a general framework for Hierarchical Image Representation Learning (HIRL). This framework aims to learn multiple semantic representations for each image, and these representations are structured to encode image semantics from fine-grained to coarse-grained. Based on a probabilistic factorization, HIRL learns the most fine-grained semantics by an off-the-shelf image SSL approach and learns multiple coarse-grained semantics by a novel semantic path discrimination scheme. We adopt six representative image SSL methods as baselines and study how they perform under HIRL. By rigorous fair comparison, performance gain is observed on all the six methods for diverse downstream tasks, which, for the first time, verifies the general effectiveness of learning hierarchical image semantics. All source code and model weights are available at https://github.com/hirl-team/HIRL

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 44 of its 50 references that are in the map. First posted on arXiv 26 May 2022.

Builds on

44 of 50 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  7. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  10. 2017Mask R-CNNcited by 7,259
  11. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  12. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
32 more
  1. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  2. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  4. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  5. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  6. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  7. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  8. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  9. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  10. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  11. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  12. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  13. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  14. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  15. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  16. 2016Colorful Image Colorizationcited by 1,113
  17. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  18. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  19. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 763
  20. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  21. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 636
  22. 2017Large Batch Training of Convolutional Networkscited by 529
  23. 2019Self-labelling via simultaneous clustering and representation learningcited by 525
  24. 2019Cascade R-CNN: High Quality Object Detection and Instance Segmentationcited by 428
  25. 2020Debiased Contrastive Learningcited by 354
  26. 2022Context Autoencoder for Self-Supervised Representation Learningcited by 297
  27. 2016Learning Representations for Automatic Colorizationcited by 251
  28. 2016Joint Unsupervised Learning of Deep Representations and Image Clusterscited by 205
  29. 2021Contrastive Learning with Stronger Augmentationscited by 100
  30. 2020AdCo: Adversarial Contrast for Efficient Learning of Unsupervised Representations from Self-Trained Negative Adversariescited by 90
  31. 2021Incremental False Negative Detection for Contrastive Learningcited by 48
  32. 2022HCSC: Hierarchical Contrastive Selective Codingcited by 43

Built on by

Not cited yet.