# Distilling Localization for Self-Supervised Representation Learning

arXiv: https://arxiv.org/abs/2004.06638 · HTML: https://mlsanity.com/paper/2004.06638/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2020.

## Built on by (most cited first, 30 of 34)

- [A Cookbook of Self-Supervised Learning](https://mlsanity.com/paper/2304.12210/) (2023)
- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [Instance Localization for Self-supervised Detection Pretraining](https://mlsanity.com/paper/2102.08318/) (2021)
- [Object discovery and representation networks](https://mlsanity.com/paper/2203.08777/) (2022)
- [Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet?](https://mlsanity.com/paper/2201.05119/) (2022)
- [Object-aware Contrastive Learning for Debiased Scene Representation](https://mlsanity.com/paper/2108.00049/) (2021)
- [Unsupervised Pre-training for Temporal Action Localization Tasks](https://mlsanity.com/paper/2203.13609/) (2022)
- [Extreme Masking for Learning Instance and Distributed Visual Representations](https://mlsanity.com/paper/2206.04667/) (2022)
- [Look at What I'm Doing: Self-Supervised Spatial Grounding of Narrations in Instructional Videos](https://mlsanity.com/paper/2110.10596/) (2021)
- [Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems](https://mlsanity.com/paper/2303.01669/) (2023)
- [Exploring the Coordination of Frequency and Attention in Masked Image Modeling](https://mlsanity.com/paper/2211.15362/) (2022)
- [ConCL: Concept Contrastive Learning for Dense Prediction Pre-training in Pathology Images](https://mlsanity.com/paper/2207.06733/) (2022)
- [Exploring Localization for Self-supervised Fine-grained Contrastive Learning](https://mlsanity.com/paper/2106.15788/) (2021)
- [Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentations](https://mlsanity.com/paper/2103.12719/) (2021)
- [SI-Score: An image dataset for fine-grained analysis of robustness to object location, rotation and size](https://mlsanity.com/paper/2104.04191/) (2021)
- [Adversarial Auto-Augment with Label Preservation: A Representation Learning Principle Guided Approach](https://mlsanity.com/paper/2211.00824/) (2022)
- [Randomized Quantization: A Generic Augmentation for Data Agnostic Self-supervised Learning](https://mlsanity.com/paper/2212.08663/) (2022)
- [CoDo: Contrastive Learning with Downstream Background Invariance for Detection](https://mlsanity.com/paper/2205.04617/) (2022)
- [Leveraging Superfluous Information in Contrastive Representation Learning](https://mlsanity.com/paper/2408.10292/) (2024)
- [Self-Supervised Learning from Unlabeled Fundus Photographs Improves Segmentation of the Retina](https://mlsanity.com/paper/2108.02798/) (2021)
- [Exemplar Learning for Medical Image Segmentation](https://mlsanity.com/paper/2204.01713/) (2022)
- [Learning the Relation between Similarity Loss and Clustering Loss in Self-Supervised Learning](https://mlsanity.com/paper/2301.03041/) (2023)
- [Rethinking the Localization in Weakly Supervised Object Localization](https://mlsanity.com/paper/2308.06161/) (2023)
- [On Learning Discriminative Features from Synthesized Data for Self-Supervised Fine-Grained Visual Recognition](https://mlsanity.com/paper/2407.14676/) (2024)
- [Cross-Level Multi-Instance Distillation for Self-Supervised Fine-Grained Visual Categorization](https://mlsanity.com/paper/2401.08860/) (2024)
- [Leveraging Self-Supervised Instance Contrastive Learning for Radar Object Detection](https://mlsanity.com/paper/2402.08427/) (2024)
- [LoDisc: Learning Global-Local Discriminative Features for Self-Supervised Fine-Grained Visual Recognition](https://mlsanity.com/paper/2403.04066/) (2024)
- [PP-SSL : Priority-Perception Self-Supervised Learning for Fine-Grained Recognition](https://mlsanity.com/paper/2412.00134/) (2024)
- [Disentangling Content from Style to Overcome Shortcut Learning: A Hybrid Generative-Discriminative Learning Framework](https://mlsanity.com/paper/2509.11598/) (2025)
- [Auto-Focus Contrastive Learning for Image Manipulation Detection](https://mlsanity.com/paper/2211.10922/) (2022)

## Abstract (arXiv)

Recent progress in contrastive learning has revolutionized unsupervised representation learning. Concretely, multiple views (augmentations) from the same image are encouraged to map to the similar embeddings, while views from different images are pulled apart. In this paper, through visualizing and diagnosing classification errors, we observe that current contrastive models are ineffective at localizing the foreground object, limiting their ability to extract discriminative high-level features. This is due to the fact that view generation process considers pixels in an image uniformly. To address this problem, we propose a data-driven approach for learning invariance to backgrounds. It first estimates foreground saliency in images and then creates augmentations by copy-and-pasting the foreground onto a variety of backgrounds. The learning still follows the instance discrimination pretext task, so that the representation is trained to disregard background content and focus on the foreground. We study a variety of saliency estimation methods, and find that most methods lead to improvements for contrastive learning. With this approach (DiLo), significant performance is achieved for self-supervised learning on ImageNet classification, and also for object detection on PASCAL VOC and MSCOCO.