# AutoFocus: Efficient Multi-Scale Inference

arXiv: https://arxiv.org/abs/1812.01600 · HTML: https://mlsanity.com/paper/1812.01600/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2018.

## Built on by (most cited first, 30 of 46)

- [AdaViT: Adaptive Vision Transformers for Efficient Image Recognition](https://mlsanity.com/paper/2111.15668/) (2021)
- [Towards Large-Scale Small Object Detection: Survey and Benchmarks](https://mlsanity.com/paper/2207.14096/) (2022)
- [QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection](https://mlsanity.com/paper/2103.09136/) (2021)
- [Rethinking Pseudo Labels for Semi-Supervised Object Detection](https://mlsanity.com/paper/2106.00168/) (2021)
- [Glance and Focus Networks for Dynamic Visual Recognition](https://mlsanity.com/paper/2201.03014/) (2022)
- [Global Features are All You Need for Image Retrieval and Reranking](https://mlsanity.com/paper/2308.06954/) (2023)
- [Disentangle Your Dense Object Detector](https://mlsanity.com/paper/2107.02963/) (2021)
- [AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognition](https://mlsanity.com/paper/2105.05165/) (2021)
- [Dynamic Network Quantization for Efficient Video Inference](https://mlsanity.com/paper/2108.10394/) (2021)
- [UFPMP-Det: Toward Accurate and Efficient Object Detection on Drone Imagery](https://mlsanity.com/paper/2112.10415/) (2021)
- [Dynamic Spatial Sparsification for Efficient Vision Transformers and Convolutional Neural Networks](https://mlsanity.com/paper/2207.01580/) (2022)
- [SILT: Shadow-aware Iterative Label Tuning for Learning to Detect Shadows from Noisy Labels](https://mlsanity.com/paper/2308.12064/) (2023)
- [Hard-Attention for Scalable Image Classification](https://mlsanity.com/paper/2102.10212/) (2021)
- [DeepPerform: An Efficient Approach for Performance Testing of Resource-Constrained Neural Networks](https://mlsanity.com/paper/2210.05370/) (2022)
- [Circle Representation for Medical Object Detection](https://mlsanity.com/paper/2110.12093/) (2021)
- [SparseDet: Improving Sparsely Annotated Object Detection with Pseudo-positive Mining](https://mlsanity.com/paper/2201.04620/) (2022)
- [AdaZoom: Adaptive Zoom Network for Multi-Scale Object Detection in Large Scenes](https://mlsanity.com/paper/2106.10409/) (2021)
- [Towards Efficient Use of Multi-Scale Features in Transformer-Based Object Detectors](https://mlsanity.com/paper/2208.11356/) (2022)
- [DyCL: Dynamic Neural Network Compilation Via Program Rewriting and Graph Optimization](https://mlsanity.com/paper/2307.04963/) (2023)
- [Is Underwater Image Enhancement All Object Detectors Need?](https://mlsanity.com/paper/2311.18814/) (2023)
- [Parameter-Inverted Image Pyramid Networks](https://mlsanity.com/paper/2406.04330/) (2024)
- [Underwater Object Detection in the Era of Artificial Intelligence: Current, Challenge, and Future](https://mlsanity.com/paper/2410.05577/) (2024)
- [Parameter-Inverted Image Pyramid Networks for Visual Perception and Multimodal Understanding](https://mlsanity.com/paper/2501.07783/) (2025)
- [Multi-Scale Fusion for Object Representation](https://mlsanity.com/paper/2410.01539/) (2024)
- [SparseFormer: Detecting Objects in HRW Shots via Sparse Vision Transformer](https://mlsanity.com/paper/2502.07216/) (2025)
- [Fewer is More: Efficient Object Detection in Large Aerial Images](https://mlsanity.com/paper/2212.13136/) (2022)
- [Scale Normalized Image Pyramids with AutoFocus for Object Detection](https://mlsanity.com/paper/2102.05646/) (2021)
- [Tackling the Background Bias in Sparse Object Detection via Cropped Windows](https://mlsanity.com/paper/2106.02288/) (2021)
- [SaccadeDet: A Novel Dual-Stage Architecture for Rapid and Accurate Detection in Gigapixel Images](https://mlsanity.com/paper/2407.17956/) (2024)
- [3D Small Object Detection with Dynamic Spatial Pruning](https://mlsanity.com/paper/2305.03716/) (2023)

## Abstract (arXiv)

This paper describes AutoFocus, an efficient multi-scale inference algorithm for deep-learning based object detectors. Instead of processing an entire image pyramid, AutoFocus adopts a coarse to fine approach and only processes regions which are likely to contain small objects at finer scales. This is achieved by predicting category agnostic segmentation maps for small objects at coarser scales, called FocusPixels. FocusPixels can be predicted with high recall, and in many cases, they only cover a small fraction of the entire image. To make efficient use of FocusPixels, an algorithm is proposed which generates compact rectangular FocusChips which enclose FocusPixels. The detector is only applied inside FocusChips, which reduces computation while processing finer scales. Different types of error can arise when detections from FocusChips of multiple scales are combined, hence techniques to correct them are proposed. AutoFocus obtains an mAP of 47.9% (68.3% at 50% overlap) on the COCO test-dev set while processing 6.4 images per second on a Titan X (Pascal) GPU. This is 2.5X faster than our multi-scale baseline detector and matches its mAP. The number of pixels processed in the pyramid can be reduced by 5X with a 1% drop in mAP. AutoFocus obtains more than 10% mAP gain compared to RetinaNet but runs at the same speed with the same ResNet-101 backbone.