# Cheaper Pre-training Lunch: An Efficient Paradigm for Object Detection

arXiv: https://arxiv.org/abs/2004.12178 · HTML: https://mlsanity.com/paper/2004.12178/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2020.

## Built on by (most cited first, 6 of 6)

- [DETReg: Unsupervised Pretraining with Region Priors for Object Detection](https://mlsanity.com/paper/2106.04550/) (2021)
- [MosaicOS: A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection](https://mlsanity.com/paper/2102.08884/) (2021)
- [Learning with Free Object Segments for Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2202.11124/) (2022)
- [DAP: Detection-Aware Pre-training with Weak Supervision](https://mlsanity.com/paper/2103.16651/) (2021)
- [Rethinking Training from Scratch for Object Detection](https://mlsanity.com/paper/2106.03112/) (2021)
- [Soft-labeling Strategies for Rapid Sub-Typing](https://mlsanity.com/paper/2209.12684/) (2022)

## Abstract (arXiv)

In this paper, we propose a general and efficient pre-training paradigm, Montage pre-training, for object detection. Montage pre-training needs only the target detection dataset while taking only 1/4 computational resources compared to the widely adopted ImageNet pre-training.To build such an efficient paradigm, we reduce the potential redundancy by carefully extracting useful samples from the original images, assembling samples in a Montage manner as input, and using an ERF-adaptive dense classification strategy for model pre-training. These designs include not only a new input pattern to improve the spatial utilization but also a novel learning objective to expand the effective receptive field of the pretrained model. The efficiency and effectiveness of Montage pre-training are validated by extensive experiments on the MS-COCO dataset, where the results indicate that the models using Montage pre-training are able to achieve on-par or even better detection performances compared with the ImageNet pre-training.