# Unsupervised Object Localization: Observing the Background to Discover Objects

arXiv: https://arxiv.org/abs/2212.07834 · HTML: https://mlsanity.com/paper/2212.07834/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 75 references that are in the map.
- First posted on arXiv 15 Dec 2022.

## Builds on (most cited first, 30 of 51)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Dense Contrastive Learning for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2011.09157/) (2020)
- [Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision](https://mlsanity.com/paper/2106.01226/) (2021)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [Unbiased Teacher for Semi-Supervised Object Detection](https://mlsanity.com/paper/2102.09480/) (2021)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Learning Representations for Automatic Colorization](https://mlsanity.com/paper/1603.06668/) (2016)
- [Unsupervised Semantic Segmentation by Distilling Feature Correspondences](https://mlsanity.com/paper/2203.08414/) (2022)
- [EGNet:Edge Guidance Network for Salient Object Detection](https://mlsanity.com/paper/1908.08297/) (2019)
- [Localizing Objects with Self-Supervised Transformers and no Labels](https://mlsanity.com/paper/2109.14279/) (2021)
- [Bridging the Gap to Real-World Object-Centric Learning](https://mlsanity.com/paper/2209.14860/) (2022)
- [A Simple Pooling-Based Design for Real-Time Salient Object Detection](https://mlsanity.com/paper/1904.09569/) (2019)
- [Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals](https://mlsanity.com/paper/2102.06191/) (2021)
- [Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cut](https://mlsanity.com/paper/2202.11539/) (2022)
- [Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization](https://mlsanity.com/paper/2205.07839/) (2022)
- [FreeSOLO: Learning to Segment Objects without Annotations](https://mlsanity.com/paper/2202.12181/) (2022)

## Built on by (most cited first, 30 of 66)

- [CLIP-DINOiser: Teaching CLIP a few DINO tricks for open-vocabulary semantic segmentation](https://mlsanity.com/paper/2312.12359/) (2023)
- [Pay Attention to Your Neighbours: Training-Free Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2404.08181/) (2024)
- [Diffusion Models for Open-Vocabulary Segmentation](https://mlsanity.com/paper/2306.09316/) (2023)
- [A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Future](https://mlsanity.com/paper/2307.09220/) (2023)
- [DiffusionSeg: Adapting Diffusion Towards Unsupervised Object Discovery](https://mlsanity.com/paper/2303.09813/) (2023)
- [SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformers](https://mlsanity.com/paper/2312.00648/) (2023)
- [Three Pillars improving Vision Foundation Model Distillation for Lidar](https://mlsanity.com/paper/2310.17504/) (2023)
- [Talking to DINO: Bridging Self-Supervised Vision Backbones with Language for Open-Vocabulary Segmentation](https://mlsanity.com/paper/2411.19331/) (2024)
- [EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Models](https://mlsanity.com/paper/2401.11739/) (2024)
- [CLIP-DIY: CLIP Dense Inference Yields Open-Vocabulary Semantic Segmentation For-Free](https://mlsanity.com/paper/2309.14289/) (2023)
- [In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2408.04961/) (2024)
- [DiffCut: Catalyzing Zero-Shot Semantic Segmentation with Diffusion Features and Recursive Normalized Cut](https://mlsanity.com/paper/2406.02842/) (2024)
- [Target before Shooting: Accurate Anomaly Detection and Localization under One Millisecond via Cascade Patch Retrieval](https://mlsanity.com/paper/2308.06748/) (2023)
- [LPOSS: Label Propagation Over Patches and Pixels for Open-vocabulary Semantic Segmentation](https://mlsanity.com/paper/2503.19777/) (2025)
- [Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Survey](https://mlsanity.com/paper/2310.12904/) (2023)
- [Near, far: Patch-ordering enhances vision foundation models' scene understanding](https://mlsanity.com/paper/2408.11054/) (2024)
- [Detect Everything with Few Examples](https://mlsanity.com/paper/2309.12969/) (2023)
- [Weakly-supervised Contrastive Learning for Unsupervised Object Discovery](https://mlsanity.com/paper/2307.03376/) (2023)
- [CamoTeacher: Dual-Rotation Consistency Learning for Semi-Supervised Camouflaged Object Detection](https://mlsanity.com/paper/2408.08050/) (2024)
- [HEAP: Unsupervised Object Discovery and Localization with Contrastive Grouping](https://mlsanity.com/paper/2312.17492/) (2023)
- [HASSOD: Hierarchical Adaptive Self-Supervised Object Detection](https://mlsanity.com/paper/2402.03311/) (2024)
- [UCOD-DPL: Unsupervised Camouflaged Object Detection via Dynamic Pseudo-label Learning](https://mlsanity.com/paper/2506.07087/) (2025)
- [PaintSeg: Training-free Segmentation via Painting](https://mlsanity.com/paper/2305.19406/) (2023)
- [SOHES: Self-supervised Open-world Hierarchical Entity Segmentation](https://mlsanity.com/paper/2404.12386/) (2024)
- [Bayesian Active Learning for Semantic Segmentation](https://mlsanity.com/paper/2408.01694/) (2024)
- [MoSiC: Optimal-Transport Motion Trajectory for Dense Self-Supervised Learning](https://mlsanity.com/paper/2506.08694/) (2025)
- [Beyond Single Images: Retrieval Self-Augmented Unsupervised Camouflaged Object Detection](https://mlsanity.com/paper/2510.18437/) (2025)
- [Unsupervised Camouflaged Object Segmentation as Domain Adaptation](https://mlsanity.com/paper/2308.04528/) (2023)
- [SEMPART: Self-supervised Multi-resolution Partitioning of Image Semantics](https://mlsanity.com/paper/2309.10972/) (2023)
- [unMORE: Unsupervised Multi-Object Segmentation via Center-Boundary Reasoning](https://mlsanity.com/paper/2506.01778/) (2025)

## Abstract (arXiv)

Recent advances in self-supervised visual representation learning have paved the way for unsupervised methods tackling tasks such as object discovery and instance segmentation. However, discovering objects in an image with no supervision is a very hard task; what are the desired objects, when to separate them into parts, how many are there, and of what classes? The answers to these questions depend on the tasks and datasets of evaluation. In this work, we take a different approach and propose to look for the background instead. This way, the salient objects emerge as a by-product without any strong assumption on what an object should be. We propose FOUND, a simple model made of a single conv1×1 initialized with coarse background masks extracted from self-supervised patch-based representations. After fast training and refining these seed masks, the model reaches state-of-the-art results on unsupervised saliency detection and object discovery benchmarks. Moreover, we show that our approach yields good results in the unsupervised semantic segmentation retrieval task. The code to reproduce our results is available at https://github.com/valeoai/FOUND.