# MegDet: A Large Mini-Batch Object Detector

arXiv: https://arxiv.org/abs/1711.07240 · HTML: https://mlsanity.com/paper/1711.07240/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2017.

## Built on by (most cited first, 30 of 66)

- [Bottleneck Transformers for Visual Recognition](https://mlsanity.com/paper/2101.11605/) (2021)
- [DetCo: Unsupervised Contrastive Learning for Object Detection](https://mlsanity.com/paper/2102.04803/) (2021)
- [RedCaps: web-curated image-text data created by the people, for the people](https://mlsanity.com/paper/2111.11431/) (2021)
- [M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representation](https://mlsanity.com/paper/2204.05088/) (2022)
- [Efficient Visual Pretraining with Contrastive Detection](https://mlsanity.com/paper/2103.10957/) (2021)
- [Cross-Domain Adaptive Teacher for Object Detection](https://mlsanity.com/paper/2111.13216/) (2021)
- [Multimodal Contrastive Training for Visual Representation Learning](https://mlsanity.com/paper/2104.12836/) (2021)
- [Aligning Bag of Regions for Open-Vocabulary Object Detection](https://mlsanity.com/paper/2302.13996/) (2023)
- [Involution: Inverting the Inherence of Convolution for Visual Recognition](https://mlsanity.com/paper/2103.06255/) (2021)
- [Region Similarity Representation Learning](https://mlsanity.com/paper/2103.12902/) (2021)
- [Object discovery and representation networks](https://mlsanity.com/paper/2203.08777/) (2022)
- [CBNet: A Composite Backbone Network Architecture for Object Detection](https://mlsanity.com/paper/2107.00420/) (2021)
- [Unsupervised Object-Level Representation Learning from Scene Images](https://mlsanity.com/paper/2106.11952/) (2021)
- [Cross Modal Transformer: Towards Fast and Robust 3D Object Detection](https://mlsanity.com/paper/2301.01283/) (2023)
- [Spatially Consistent Representation Learning](https://mlsanity.com/paper/2103.06122/) (2021)
- [ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training](https://mlsanity.com/paper/2104.14129/) (2021)
- [Dissecting Self-Supervised Learning Methods for Surgical Computer Vision](https://mlsanity.com/paper/2207.00449/) (2022)
- [Rethinking "Batch" in BatchNorm](https://mlsanity.com/paper/2105.07576/) (2021)
- [Improving Contrastive Learning by Visualizing Feature Transformation](https://mlsanity.com/paper/2108.02982/) (2021)
- [Robust and Accurate Object Detection via Adversarial Learning](https://mlsanity.com/paper/2103.13886/) (2021)
- [Effective Adaptation in Multi-Task Co-Training for Unified Autonomous Driving](https://mlsanity.com/paper/2209.08953/) (2022)
- [Semantic Diffusion Network for Semantic Segmentation](https://mlsanity.com/paper/2302.02057/) (2023)
- [GraphFPN: Graph Feature Pyramid Network for Object Detection](https://mlsanity.com/paper/2108.00580/) (2021)
- [Instance Shadow Detection with A Single-Stage Detector](https://mlsanity.com/paper/2207.04614/) (2022)
- [Momentum^2 Teacher: Momentum Teacher with Momentum Statistics for Self-Supervised Learning](https://mlsanity.com/paper/2101.07525/) (2021)
- [Adaptable Deformable Convolutions for Semantic Segmentation of Fisheye Images in Autonomous Driving Systems](https://mlsanity.com/paper/2102.10191/) (2021)
- [Quality-Aware Network for Human Parsing](https://mlsanity.com/paper/2103.05997/) (2021)
- [Rethinking Performance Gains in Image Dehazing Networks](https://mlsanity.com/paper/2209.11448/) (2022)
- [USB: Universal-Scale Object Detection Benchmark](https://mlsanity.com/paper/2103.14027/) (2021)
- [Efficient Global-Local Memory for Real-time Instrument Segmentation of Robotic Surgical Video](https://mlsanity.com/paper/2109.13593/) (2021)

## Abstract (arXiv)

The improvements in recent CNN-based object detection works, from R-CNN [11], Fast/Faster R-CNN [10, 31] to recent Mask R-CNN [14] and RetinaNet [24], mainly come from new network, new framework, or novel loss design. But mini-batch size, a key factor in the training, has not been well studied. In this paper, we propose a Large MiniBatch Object Detector (MegDet) to enable the training with much larger mini-batch size than before (e.g. from 16 to 256), so that we can effectively utilize multiple GPUs (up to 128 in our experiments) to significantly shorten the training time. Technically, we suggest a learning rate policy and Cross-GPU Batch Normalization, which together allow us to successfully train a large mini-batch detector in much less time (e.g., from 33 hours to 4 hours), and achieve even better accuracy. The MegDet is the backbone of our submission (mmAP 52.5%) to COCO 2017 Challenge, where we won the 1st place of Detection task.