# FreeSOLO: Learning to Segment Objects without Annotations

arXiv: https://arxiv.org/abs/2202.12181 · HTML: https://mlsanity.com/paper/2202.12181/

## Key facts

- Cited by 102 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 74 references that are in the map.
- First posted on arXiv 24 Feb 2022.

## Builds on (most cited first, 30 of 57)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation](https://mlsanity.com/paper/2012.07177/) (2020)
- [Dense Contrastive Learning for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2011.09157/) (2020)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [Conditional Convolutions for Instance Segmentation](https://mlsanity.com/paper/2003.05664/) (2020)
- [Contrastive learning of global and local features for medical image segmentation with limited annotations](https://mlsanity.com/paper/2006.10511/) (2020)
- [Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/2011.10043/) (2020)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Localizing Objects with Self-Supervised Transformers and no Labels](https://mlsanity.com/paper/2109.14279/) (2021)

## Built on by (most cited first, 30 of 102)

- [Cut and Learn for Unsupervised Object Detection and Instance Segmentation](https://mlsanity.com/paper/2301.11320/) (2023)
- [Transformer-Based Visual Segmentation: A Survey](https://mlsanity.com/paper/2304.09854/) (2023)
- [MinVIS: A Minimal Video Instance Segmentation Framework without Video-based Training](https://mlsanity.com/paper/2208.02245/) (2022)
- [Self-Supervised Visual Representation Learning with Semantic Grouping](https://mlsanity.com/paper/2205.15288/) (2022)
- [CLIP as RNN: Segment Countless Visual Concepts without Training Endeavor](https://mlsanity.com/paper/2312.07661/) (2023)
- [Unsupervised Object Localization: Observing the Background to Discover Objects](https://mlsanity.com/paper/2212.07834/) (2022)
- [Diffusion Models for Open-Vocabulary Segmentation](https://mlsanity.com/paper/2306.09316/) (2023)
- [Zero-shot Referring Image Segmentation with Global-Local Context Features](https://mlsanity.com/paper/2303.17811/) (2023)
- [DiffusionSeg: Adapting Diffusion Towards Unsupervised Object Discovery](https://mlsanity.com/paper/2303.09813/) (2023)
- [A Survey on Label-efficient Deep Image Segmentation: Bridging the Gap between Weak Supervision and Dense Prediction](https://mlsanity.com/paper/2207.01223/) (2022)
- [ACSeg: Adaptive Conceptualization for Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2210.05944/) (2022)
- [CLIP-DIY: CLIP Dense Inference Yields Open-Vocabulary Semantic Segmentation For-Free](https://mlsanity.com/paper/2309.14289/) (2023)
- [In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2408.04961/) (2024)
- [VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentation](https://mlsanity.com/paper/2308.14710/) (2023)
- [Unsupervised Universal Image Segmentation](https://mlsanity.com/paper/2312.17243/) (2023)
- [Adaptive Slot Attention: Object Discovery with Dynamic Slot Number](https://mlsanity.com/paper/2406.09196/) (2024)
- [Ref-Diff: Zero-shot Referring Image Segmentation with Generative Models](https://mlsanity.com/paper/2308.16777/) (2023)
- [BoxTeacher: Exploring High-Quality Pseudo Labels for Weakly Supervised Instance Segmentation](https://mlsanity.com/paper/2210.05174/) (2022)
- [UnScene3D: Unsupervised 3D Instance Segmentation for Indoor Scenes](https://mlsanity.com/paper/2303.14541/) (2023)
- [Segment Anything without Supervision](https://mlsanity.com/paper/2406.20081/) (2024)
- [IteRPrimE: Zero-shot Referring Image Segmentation with Iterative Grad-CAM Refinement and Primary Word Emphasis](https://mlsanity.com/paper/2503.00936/) (2025)
- [CL-MAE: Curriculum-Learned Masked Autoencoders](https://mlsanity.com/paper/2308.16572/) (2023)
- [UNION: Unsupervised 3D Object Detection using Object Appearance-based Pseudo-Classes](https://mlsanity.com/paper/2405.15688/) (2024)
- [OV-DQUO: Open-Vocabulary DETR with Denoising Text Query Training and Open-World Unknown Objects Supervision](https://mlsanity.com/paper/2405.17913/) (2024)
- [CrOC: Cross-View Online Clustering for Dense Visual Representation Learning](https://mlsanity.com/paper/2303.13245/) (2023)
- [Text Augmented Spatial-aware Zero-shot Referring Image Segmentation](https://mlsanity.com/paper/2310.18049/) (2023)
- [MOVE: Unsupervised Movable Object Segmentation and Detection](https://mlsanity.com/paper/2210.07920/) (2022)
- [4D Unsupervised Object Discovery](https://mlsanity.com/paper/2210.04801/) (2022)
- [Few-Shot Panoptic Segmentation With Foundation Models](https://mlsanity.com/paper/2309.10726/) (2023)
- [SmooSeg: Smoothness Prior for Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2310.17874/) (2023)

## Abstract (arXiv)

Instance segmentation is a fundamental vision task that aims to recognize and segment each object in an image. However, it requires costly annotations such as bounding boxes and segmentation masks for learning. In this work, we propose a fully unsupervised learning method that learns class-agnostic instance segmentation without any annotations. We present FreeSOLO, a self-supervised instance segmentation framework built on top of the simple instance segmentation method SOLO. Our method also presents a novel localization-aware pre-training framework, where objects can be discovered from complicated scenes in an unsupervised manner. FreeSOLO achieves 9.8% AP_{50} on the challenging COCO dataset, which even outperforms several segmentation proposal methods that use manual annotations. For the first time, we demonstrate unsupervised class-agnostic instance segmentation successfully. FreeSOLO's box localization significantly outperforms state-of-the-art unsupervised object detection/discovery methods, with about 100% relative improvements in COCO AP. FreeSOLO further demonstrates superiority as a strong pre-training method, outperforming state-of-the-art self-supervised pre-training methods by +9.8% AP when fine-tuning instance segmentation with only 5% COCO masks. Code is available at: github.com/NVlabs/FreeSOLO