# Scale-aware Automatic Augmentation for Object Detection

arXiv: https://arxiv.org/abs/2103.17220 · HTML: https://mlsanity.com/paper/2103.17220/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 52 references that are in the map.
- First posted on arXiv 31 Mar 2021.

## Builds on (most cited first, 30 of 43)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection](https://mlsanity.com/paper/1912.02424/) (2019)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [Soft-NMS -- Improving Object Detection With One Line of Code](https://mlsanity.com/paper/1704.04503/) (2017)
- [The Effectiveness of Data Augmentation in Image Classification using Deep Learning](https://mlsanity.com/paper/1712.04621/) (2017)
- [DropBlock: A regularization method for convolutional networks](https://mlsanity.com/paper/1810.12890/) (2018)
- [Fast AutoAugment](https://mlsanity.com/paper/1905.00397/) (2019)
- [Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection](https://mlsanity.com/paper/1708.01642/) (2017)
- [Scale-Aware Trident Networks for Object Detection](https://mlsanity.com/paper/1901.01892/) (2019)
- [Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules](https://mlsanity.com/paper/1905.05393/) (2019)
- [Learning Data Augmentation Strategies for Object Detection](https://mlsanity.com/paper/1906.11172/) (2019)
- [GridMask Data Augmentation](https://mlsanity.com/paper/2001.04086/) (2020)
- [An Analysis of Scale Invariance in Object Detection - SNIP](https://mlsanity.com/paper/1711.08189/) (2017)

## Built on by (most cited first, 23 of 23)

- [DAMO-YOLO : A Report on Real-Time Object Detection Design](https://mlsanity.com/paper/2211.15444/) (2022)
- [AcroFOD: An Adaptive Method for Cross-domain Few-shot Object Detection](https://mlsanity.com/paper/2209.10904/) (2022)
- [Multi-Scale Aligned Distillation for Low-Resolution Detection](https://mlsanity.com/paper/2109.06875/) (2021)
- [Image Data Augmentation Approaches: A Comprehensive Survey and Future directions](https://mlsanity.com/paper/2301.02830/) (2023)
- [DiffusionEngine: Diffusion Model is Scalable Data Engine for Object Detection](https://mlsanity.com/paper/2309.03893/) (2023)
- [MoCaE: Mixture of Calibrated Experts Significantly Improves Object Detection](https://mlsanity.com/paper/2309.14976/) (2023)
- [A Survey of Automated Data Augmentation Algorithms for Deep Learning-based Image Classification Tasks](https://mlsanity.com/paper/2206.06544/) (2022)
- [Improved YOLOv5 network for real-time multi-scale traffic sign detection](https://mlsanity.com/paper/2112.08782/) (2021)
- [Trapped in texture bias? A large scale comparison of deep instance segmentation](https://mlsanity.com/paper/2401.09109/) (2024)
- [CA-SSL: Class-Agnostic Semi-Supervised Learning for Detection and Segmentation](https://mlsanity.com/paper/2112.04966/) (2021)
- [Improving Crowded Object Detection via Copy-Paste](https://mlsanity.com/paper/2211.12110/) (2022)
- [AIMS: All-Inclusive Multi-Level Segmentation](https://mlsanity.com/paper/2305.17768/) (2023)
- [ASAG: Building Strong One-Decoder-Layer Sparse Detectors via Adaptive Sparse Anchor Generation](https://mlsanity.com/paper/2308.09242/) (2023)
- [Bootstrap Your Object Detector via Mixed Training](https://mlsanity.com/paper/2111.03056/) (2021)
- [GPTR: Gestalt-Perception Transformer for Diagram Object Detection](https://mlsanity.com/paper/2212.14232/) (2022)
- [Supporting Safety Analysis of Image-processing DNNs through Clustering-based Approaches](https://mlsanity.com/paper/2301.13506/) (2023)
- [Data Augmentation in Human-Centric Vision](https://mlsanity.com/paper/2403.08650/) (2024)
- [DA-Mamba: Domain Adaptive Hybrid Mamba-Transformer Based One-Stage Object Detection](https://mlsanity.com/paper/2502.11178/) (2025)
- [Towards Effective Collaborative Learning in Long-Tailed Recognition](https://mlsanity.com/paper/2305.03378/) (2023)
- [DyRA: Portable Dynamic Resolution Adjustment Network for Existing Detectors](https://mlsanity.com/paper/2311.17098/) (2023)
- [CIA: Controllable Image Augmentation Framework Based on Stable Diffusion](https://mlsanity.com/paper/2411.16128/) (2024)
- [Efficient Feature Aggregation and Scale-Aware Regression for Monocular 3D Object Detection](https://mlsanity.com/paper/2411.02747/) (2024)
- [Topology-Preserving Polygon Augmentation for Segmentation in Structured Visual Domains](https://mlsanity.com/paper/2603.14764/) (2026)

## Abstract (arXiv)

We propose Scale-aware AutoAug to learn data augmentation policies for object detection. We define a new scale-aware search space, where both image- and box-level augmentations are designed for maintaining scale invariance. Upon this search space, we propose a new search metric, termed Pareto Scale Balance, to facilitate search with high efficiency. In experiments, Scale-aware AutoAug yields significant and consistent improvement on various object detectors (e.g., RetinaNet, Faster R-CNN, Mask R-CNN, and FCOS), even compared with strong multi-scale training baselines. Our searched augmentation policies are transferable to other datasets and box-level tasks beyond object detection (e.g., instance segmentation and keypoint estimation) to improve performance. The search cost is much less than previous automated augmentation approaches for object detection. It is notable that our searched policies have meaningful patterns, which intuitively provide valuable insight for human data augmentation design. Code and models will be available at https://github.com/Jia-Research-Lab/SA-AutoAug.