# Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision

arXiv: https://arxiv.org/abs/2104.01257 · HTML: https://mlsanity.com/paper/2104.01257/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 49 references that are in the map.
- First posted on arXiv 2 Apr 2021.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [Hyperbolic Neural Networks](https://mlsanity.com/paper/1805.09112/) (2018)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [Learning Representations for Automatic Colorization](https://mlsanity.com/paper/1603.06668/) (2016)
- [BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1503.01640/) (2015)
- [Simple Does It: Weakly Supervised Instance and Semantic Segmentation](https://mlsanity.com/paper/1603.07485/) (2016)
- [Hyperbolic Image Embeddings](https://mlsanity.com/paper/1904.02239/) (2019)
- [Colorization as a Proxy Task for Visual Understanding](https://mlsanity.com/paper/1703.04044/) (2017)
- [Simultaneous Detection and Segmentation](https://mlsanity.com/paper/1407.1808/) (2014)
- [Semantic Instance Segmentation with a Discriminative Loss Function](https://mlsanity.com/paper/1708.02551/) (2017)
- [Continuous Hierarchical Representations with Poincar'e Variational Auto-Encoders](https://mlsanity.com/paper/1901.06033/) (2019)
- [SCOPS: Self-Supervised Co-Part Segmentation](https://mlsanity.com/paper/1905.01298/) (2019)
- [Weakly Supervised Instance Segmentation using Class Peak Response](https://mlsanity.com/paper/1804.00880/) (2018)
- [Unsupervised Object Discovery and Localization in the Wild: Part-based Matching with Bottom-up Region Proposals](https://mlsanity.com/paper/1501.06170/) (2015)
- [Learning to Segment Every Thing](https://mlsanity.com/paper/1711.10370/) (2017)
- [Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagery](https://mlsanity.com/paper/1711.09082/) (2017)

## Built on by (most cited first, 30 of 39)

- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [Hyperbolic Image Segmentation](https://mlsanity.com/paper/2203.05898/) (2022)
- [Self-Supervised Aggregation of Diverse Experts for Test-Agnostic Long-Tailed Recognition](https://mlsanity.com/paper/2107.09249/) (2021)
- [Hyperbolic Deep Learning in Computer Vision: A Survey](https://mlsanity.com/paper/2305.06611/) (2023)
- [Hyperbolic Contrastive Learning for Visual Representations beyond Objects](https://mlsanity.com/paper/2212.00653/) (2022)
- [CO-SNE: Dimensionality Reduction and Visualization for Hyperbolic Data](https://mlsanity.com/paper/2111.15037/) (2021)
- [A Systematic Review on Long-Tailed Learning](https://mlsanity.com/paper/2408.00483/) (2024)
- [Hyperbolic Fine-Tuning for Large Language Models](https://mlsanity.com/paper/2410.04010/) (2024)
- [ProCo: Prototype-aware Contrastive Learning for Long-tailed Medical Image Classification](https://mlsanity.com/paper/2209.00183/) (2022)
- [Poincar'e ResNet](https://mlsanity.com/paper/2303.14027/) (2023)
- [Hyperbolic Category Discovery](https://mlsanity.com/paper/2504.06120/) (2025)
- [Novel Class Discovery for Long-tailed Recognition](https://mlsanity.com/paper/2308.02989/) (2023)
- [Rethinking the compositionality of point clouds through regularization in the hyperbolic space](https://mlsanity.com/paper/2209.10318/) (2022)
- [Towards Calibrated Hyper-Sphere Representation via Distribution Overlap Coefficient for Long-tailed Learning](https://mlsanity.com/paper/2208.10043/) (2022)
- [Lorentzian Residual Neural Networks](https://mlsanity.com/paper/2412.14695/) (2024)
- [Hyperbolic Contrastive Learning](https://mlsanity.com/paper/2302.01409/) (2023)
- [HyperCore: The Core Framework for Building Hyperbolic Foundation Models with Comprehensive Modules](https://mlsanity.com/paper/2504.08912/) (2025)
- [Learning to Discover and Detect Objects](https://mlsanity.com/paper/2210.10774/) (2022)
- [Improving Visual Recognition with Hyperbolical Visual Hierarchy Mapping](https://mlsanity.com/paper/2404.00974/) (2024)
- [Understanding Hyperbolic Metric Learning through Hard Negative Sampling](https://mlsanity.com/paper/2404.15523/) (2024)
- [Lidar Panoptic Segmentation in an Open World](https://mlsanity.com/paper/2409.14273/) (2024)
- [HP-Capsule: Unsupervised Face Part Discovery by Hierarchical Parsing Capsule Network](https://mlsanity.com/paper/2203.10699/) (2022)
- [Hyperbolic Geometry in Computer Vision: A Survey](https://mlsanity.com/paper/2304.10764/) (2023)
- [Hyp-OC: Hyperbolic One Class Classification for Face Anti-Spoofing](https://mlsanity.com/paper/2404.14406/) (2024)
- [Adversarial Attacks on Hyperbolic Networks](https://mlsanity.com/paper/2412.01495/) (2024)
- [Unsupervised Feature Learning with Emergent Data-Driven Prototypicality](https://mlsanity.com/paper/2307.01421/) (2023)
- [Enhancing Novel Object Detection via Cooperative Foundational Models](https://mlsanity.com/paper/2311.12068/) (2023)
- [PANDAS: Prototype-based Novel Class Discovery and Detection](https://mlsanity.com/paper/2402.17420/) (2024)
- [Open-World Panoptic Segmentation](https://mlsanity.com/paper/2412.12740/) (2024)
- [Generalized Class Discovery in Instance Segmentation](https://mlsanity.com/paper/2502.08149/) (2025)

## Abstract (arXiv)

Instance segmentation is an active topic in computer vision that is usually solved by using supervised learning approaches over very large datasets composed of object level masks. Obtaining such a dataset for any new domain can be very expensive and time-consuming. In addition, models trained on certain annotated categories do not generalize well to unseen objects. The goal of this paper is to propose a method that can perform unsupervised discovery of long-tail categories in instance segmentation, through learning instance embeddings of masked regions. Leveraging rich relationship and hierarchical structure between objects in the images, we propose self-supervised losses for learning mask embeddings. Trained on COCO dataset without additional annotations of the long-tail objects, our model is able to discover novel and more fine-grained objects than the common categories in COCO. We show that the model achieves competitive quantitative results on LVIS as compared to the supervised and partially supervised methods.