# Self-taught Object Localization with Deep Networks

arXiv: https://arxiv.org/abs/1409.3964 · HTML: https://mlsanity.com/paper/1409.3964/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Sep 2014.

## Built on by (most cited first, 13 of 13)

- [Localizing Objects with Self-Supervised Transformers and no Labels](https://mlsanity.com/paper/2109.14279/) (2021)
- [Weakly Supervised Object Localization and Detection: A Survey](https://mlsanity.com/paper/2104.07918/) (2021)
- [Class-aware Sounding Objects Localization via Audiovisual Correspondence](https://mlsanity.com/paper/2112.11749/) (2021)
- [Online Refinement of Low-level Feature Based Activation Map for Weakly Supervised Object Localization](https://mlsanity.com/paper/2110.05741/) (2021)
- [Self-supervised object detection from audio-visual correspondence](https://mlsanity.com/paper/2104.06401/) (2021)
- [Visual Search at Alibaba](https://mlsanity.com/paper/2102.04674/) (2021)
- [Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models](https://mlsanity.com/paper/2311.17095/) (2023)
- [Self-learning for weakly supervised Gleason grading of local patterns](https://mlsanity.com/paper/2105.10420/) (2021)
- [Weakly Supervised Volumetric Segmentation via Self-taught Shape Denoising Model](https://mlsanity.com/paper/2104.13082/) (2021)
- [Learning a Weight Map for Weakly-Supervised Localization](https://mlsanity.com/paper/2111.14131/) (2021)
- [Statistical Attention Localization (SAL): Methodology and Application to Object Classification](https://mlsanity.com/paper/2208.01823/) (2022)
- [Model Interpretability and Rationale Extraction by Input Mask Optimization](https://mlsanity.com/paper/2508.11388/) (2025)
- [Weakly Supervised Virus Capsid Detection with Image-Level Annotations in Electron Microscopy Images](https://mlsanity.com/paper/2508.00563/) (2025)

## Abstract (arXiv)

This paper introduces self-taught object localization, a novel approach that leverages deep convolutional networks trained for whole-image recognition to localize objects in images without additional human supervision, i.e., without using any ground-truth bounding boxes for training. The key idea is to analyze the change in the recognition scores when artificially masking out different regions of the image. The masking out of a region that includes the object typically causes a significant drop in recognition score. This idea is embedded into an agglomerative clustering technique that generates self-taught localization hypotheses. Our object localization scheme outperforms existing proposal methods in both precision and recall for small number of subwindow proposals (e.g., on ILSVRC-2012 it produces a relative gain of 23.4% over the state-of-the-art for top-1 hypothesis). Furthermore, our experiments show that the annotations automatically-generated by our method can be used to train object detectors yielding recognition results remarkably close to those obtained by training on manually-annotated bounding boxes.