# ScratchDet: Training Single-Shot Object Detectors from Scratch

arXiv: https://arxiv.org/abs/1810.08425 · HTML: https://mlsanity.com/paper/1810.08425/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2018.

## Built on by (most cited first, 19 of 19)

- [Class-Aware Robust Adversarial Training for Object Detection](https://mlsanity.com/paper/2103.16148/) (2021)
- [Prompt-Matched Semantic Segmentation](https://mlsanity.com/paper/2208.10159/) (2022)
- [AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection](https://mlsanity.com/paper/2106.05499/) (2021)
- [EmergencyNet: Efficient Aerial Image Classification for Drone-Based Emergency Monitoring Using Atrous Convolutional Feature Fusion](https://mlsanity.com/paper/2104.14006/) (2021)
- [Dense outlier detection and open-set recognition based on training with noisy negative images](https://mlsanity.com/paper/2101.09193/) (2021)
- [Circle Representation for Medical Object Detection](https://mlsanity.com/paper/2110.12093/) (2021)
- [Deep transfer learning for image classification: a survey](https://mlsanity.com/paper/2205.09904/) (2022)
- [VIPriors 1: Visual Inductive Priors for Data-Efficient Deep Learning Challenges](https://mlsanity.com/paper/2103.03768/) (2021)
- [DAP: Detection-Aware Pre-training with Weak Supervision](https://mlsanity.com/paper/2103.16651/) (2021)
- [Few-Shot Video Object Detection](https://mlsanity.com/paper/2104.14805/) (2021)
- [Revisiting the Loss Weight Adjustment in Object Detection](https://mlsanity.com/paper/2103.09488/) (2021)
- [Knowledge Distillation for Object Detection: from generic to remote sensing datasets](https://mlsanity.com/paper/2307.09264/) (2023)
- [Data exploitation: multi-task learning of object detection and semantic segmentation on partially annotated data](https://mlsanity.com/paper/2311.04040/) (2023)
- [Non-binary deep transfer learning for image classification](https://mlsanity.com/paper/2107.08585/) (2021)
- [Self-Training and Multi-Task Learning for Limited Data: Evaluation Study on Object Detection](https://mlsanity.com/paper/2309.06288/) (2023)
- [Delving into Variance Transmission and Normalization: Shift of Average Gradient Makes the Network Collapse](https://mlsanity.com/paper/2103.11590/) (2021)
- [EMDFNet: Efficient Multi-scale and Diverse Feature Network for Traffic Sign Detection](https://mlsanity.com/paper/2408.14189/) (2024)
- [SMDDFNet: State-space Modeling and Dynamic Dual Fusion Network for Traffic Sign Detection](https://mlsanity.com/paper/2505.05491/) (2025)
- [Data-Efficient Challenges in Visual Inductive Priors: A Retrospective](https://mlsanity.com/paper/2506.08612/) (2025)

## Abstract (arXiv)

Current state-of-the-art object objectors are fine-tuned from the off-the-shelf networks pretrained on large-scale classification dataset ImageNet, which incurs some additional problems: 1) The classification and detection have different degrees of sensitivity to translation, resulting in the learning objective bias; 2) The architecture is limited by the classification network, leading to the inconvenience of modification. To cope with these problems, training detectors from scratch is a feasible solution. However, the detectors trained from scratch generally perform worse than the pretrained ones, even suffer from the convergence issue in training. In this paper, we explore to train object detectors from scratch robustly. By analysing the previous work on optimization landscape, we find that one of the overlooked points in current trained-from-scratch detector is the BatchNorm. Resorting to the stable and predictable gradient brought by BatchNorm, detectors can be trained from scratch stably while keeping the favourable performance independent to the network architecture. Taking this advantage, we are able to explore various types of networks for object detection, without suffering from the poor convergence. By extensive experiments and analyses on downsampling factor, we propose the Root-ResNet backbone network, which makes full use of the information from original images. Our ScratchDet achieves the state-of-the-art accuracy on PASCAL VOC 2007, 2012 and MS COCO among all the train-from-scratch detectors and even performs better than several one-stage pretrained methods. Codes will be made publicly available at https://github.com/KimSoybean/ScratchDet.