mlsanityData to 7 Oct 2026

Simple Training Strategies and Model Scaling for Object Detection

arXiv 2107.00057 ↗First posted 30 Jun 2021cs.CV cs.AIcited by 25

Not on the map. Why?

Abstract (arXiv)

The speed-accuracy Pareto curve of object detection systems have advanced through a combination of better model architectures, training and inference methods. In this paper, we methodically evaluate a variety of these techniques to understand where most of the improvements in modern detection systems come from. We benchmark these improvements on the vanilla ResNet-FPN backbone with RetinaNet and RCNN detectors. The vanilla detectors are improved by 7.7% in accuracy while being 30% faster in speed. We further provide simple scaling strategies to generate family of models that form two Pareto curves, named RetinaNet-RS and Cascade RCNN-RS. These simple rescaled detectors explore the speed-accuracy trade-off between the one-stage RetinaNet detectors and two-stage RCNN detectors. Our largest Cascade RCNN-RS models achieve 52.9% AP with a ResNet152-FPN backbone and 53.6% with a SpineNet143L backbone. Finally, we show the ResNet architecture, with three minor architectural changes, outperforms EfficientNet as the backbone for object detection and instance segmentation systems.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 34 of its 37 references that are in the map. First posted on arXiv 30 Jun 2021.

Builds on

34 of 37 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2017Mask R-CNNcited by 7,259
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  8. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  10. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  11. 2017Squeeze-and-Excitation Networkscited by 4,763
  12. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
22 more
  1. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  2. 2015Fast R-CNNcited by 2,930
  3. 2018YOLOv3: An Incremental Improvementcited by 2,652
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  5. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  6. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  7. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  8. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  9. 2019Objects as Pointscited by 1,172
  10. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
  11. 2016Deep Networks with Stochastic Depthcited by 973
  12. 2018Path Aggregation Network for Instance Segmentationcited by 796
  13. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  14. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 595
  15. 2019Hybrid Task Cascade for Instance Segmentationcited by 479
  16. 2018Bag of Tricks for Image Classification with Convolutional Neural Networkscited by 430
  17. 2019NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detectioncited by 289
  18. 2020Scaled-YOLOv4: Scaling Cross Stage Partial Networkcited by 211
  19. 2016Speed/accuracy trade-offs for modern convolutional object detectorscited by 198
  20. 2021Revisiting ResNets: Improved Training and Scaling Strategiescited by 158
  21. 2019SpineNet: Learning Scale-Permuted Backbone for Recognition and Localizationcited by 50
  22. 2020Efficient Scale-Permuted Backbone with Learned Resource Distributioncited by 3

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 7; 2023: 10; 2024: 2; 2025: 3.

25 papers.

Sort:

  1. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  2. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  3. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 341
  4. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 327
  5. 2022RbA: Segmenting Unknown Regions Rejected by Allcited by 67
  6. 2023Towards Open-Vocabulary Video Instance Segmentationcited by 45
  7. 2023Unmasking Anomalies in Road-Scene Segmentationcited by 45
  8. 2023A General-Purpose Self-Supervised Model for Computational Pathologycited by 42
  9. 2023MP-Former: Mask-Piloted Transformer for Image Segmentationcited by 39
  10. 2024UniVS: Unified and Universal Video Segmentation with Prompts as Queriescited by 26
  11. 2022Proper Reuse of Image Classification Features Improves Object Detectioncited by 20
  12. 2023Mask2Anomaly: Mask Transformer for Universal Open-set Segmentationcited by 18
13 more
  1. 2021Contextualized Spatio-Temporal Contrastive Learning with Self-Supervisioncited by 15
  2. 2023Compositor: Bottom-up Clustering and Compositing for Robust Part and Object Segmentationcited by 11
  3. 2021Revisiting 3D ResNets for Video Recognitioncited by 10
  4. 2023AutoFocusFormer: Image Segmentation off the Gridcited by 9
  5. 2022Learning with Free Object Segments for Long-Tailed Instance Segmentationcited by 8
  6. 2023MaskConver: Revisiting Pure Convolution Model for Panoptic Segmentationcited by 4
  7. 2025Mixtraining: A Better Trade-Off Between Compute and Performancecited by 2
  8. 2025Seg-VAR: Image Segmentation with Visual Autoregressive Modelingcited by 2
  9. 2023Training Strategies for Vision Transformers for Object Detectioncited by 1
  10. 2022Optimizing Anchor-based Detectors for Autonomous Driving Scenescited by 0
  11. 2023HAISTA-NET: Human Assisted Instance Segmentation Through Attentioncited by 0
  12. 2024From Pixels to Objects: A Hierarchical Approach for Part and Object Segmentation Using Local and Global Aggregationcited by 0
  13. 2025Transformer-Based Dual-Optical Attention Fusion Crowd Head Point Counting and Localization Networkcited by 0