# Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search

arXiv: https://arxiv.org/abs/2102.04700 · HTML: https://mlsanity.com/paper/2102.04700/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 34 references that are in the map.
- First posted on arXiv 9 Feb 2021.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection](https://mlsanity.com/paper/1912.02424/) (2019)
- [Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression](https://mlsanity.com/paper/1911.08287/) (2019)
- [Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection](https://mlsanity.com/paper/2006.04388/) (2020)
- [UnitBox: An Advanced Object Detection Network](https://mlsanity.com/paper/1608.01471/) (2016)
- [Acquisition of Localization Confidence for Accurate Object Detection](https://mlsanity.com/paper/1807.11590/) (2018)
- [AutoML-Zero: Evolving Machine Learning Algorithms From Scratch](https://mlsanity.com/paper/2003.03384/) (2020)
- [Gradient Harmonized Single-stage Detector](https://mlsanity.com/paper/1811.05181/) (2018)
- [Evolving Normalization-Activation Layers](https://mlsanity.com/paper/2004.02967/) (2020)
- [Are Labels Necessary for Neural Architecture Search?](https://mlsanity.com/paper/2003.12056/) (2020)
- [Towards Accurate One-Stage Object Detection with AP-Loss](https://mlsanity.com/paper/1904.06373/) (2019)
- [IoU-aware Single-stage Object Detector for Accurate Localization](https://mlsanity.com/paper/1912.05992/) (2019)
- [Improving Object Localization with Fitness NMS and Bounded IoU Loss](https://mlsanity.com/paper/1711.00164/) (2017)
- [AM-LFS: AutoML for Loss Function Search](https://mlsanity.com/paper/1905.07375/) (2019)
- [Improved Training Speed, Accuracy, and Data Utilization Through Loss Function Optimization](https://mlsanity.com/paper/1905.11528/) (2019)
- [DR Loss: Improving Object Detection by Distributional Ranking](https://mlsanity.com/paper/1907.10156/) (2019)
- [Loss Function Search for Face Recognition](https://mlsanity.com/paper/2007.06542/) (2020)

## Built on by (most cited first, 20 of 20)

- [Rank & Sort Loss for Object Detection and Instance Segmentation](https://mlsanity.com/paper/2107.11669/) (2021)
- [EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantization](https://mlsanity.com/paper/2307.10554/) (2023)
- [AutoLoss-Zero: Searching Loss Functions from Scratch for Generic Tasks](https://mlsanity.com/paper/2103.14026/) (2021)
- [Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression](https://mlsanity.com/paper/2110.13675/) (2021)
- [AutoLossGen: Automatic Loss Function Generation for Recommender Systems](https://mlsanity.com/paper/2204.13160/) (2022)
- [Learning Symbolic Model-Agnostic Loss Functions via Meta-Learning](https://mlsanity.com/paper/2209.08907/) (2022)
- [Meta-Learning Adaptive Loss Functions](https://mlsanity.com/paper/2301.13247/) (2023)
- [Meta-tuning Loss Functions and Data Augmentation for Few-shot Object Detection](https://mlsanity.com/paper/2304.12161/) (2023)
- [Tightening the Approximation Error of Adversarial Risk with Auto Loss Function Search](https://mlsanity.com/paper/2111.05063/) (2021)
- [Problem Learning: Towards the Free Will of Machines](https://mlsanity.com/paper/2109.00177/) (2021)
- [HLIC: Harmonizing Optimization Metrics in Learned Image Compression by Reinforcement Learning](https://mlsanity.com/paper/2109.14863/) (2021)
- [Neural Loss Function Evolution for Large-Scale Image Classifier Convolutional Neural Networks](https://mlsanity.com/paper/2403.08793/) (2024)
- [Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving LLM Agents](https://mlsanity.com/paper/2607.12790/) (2026)
- [Searching Parameterized AP Loss for Object Detection](https://mlsanity.com/paper/2112.05138/) (2021)
- [Domain Adaptable Fine-Tune Distillation Framework For Advancing Farm Surveillance](https://mlsanity.com/paper/2402.07059/) (2024)
- [Fast and Efficient Local Search for Genetic Programming Based Loss Function Learning](https://mlsanity.com/paper/2403.00865/) (2024)
- [Automated Loss function Search for Class-imbalanced Node Classification](https://mlsanity.com/paper/2405.14133/) (2024)
- [Evolving Loss Functions for Specific Image Augmentation Techniques](https://mlsanity.com/paper/2404.06633/) (2024)
- [Neural Optimizer Equation, Decay Function, and Learning Rate Schedule Joint Evolution](https://mlsanity.com/paper/2404.06679/) (2024)
- [EvoRank: LLM-Guided Evolution of Multi-Objective Learning-to-Rank Pipelines](https://mlsanity.com/paper/2609.22196/) (2026)

## Abstract (arXiv)

Designing proper loss functions for vision tasks has been a long-standing research direction to advance the capability of existing models. For object detection, the well-established classification and regression loss functions have been carefully designed by considering diverse learning challenges. Inspired by the recent progress in network architecture search, it is interesting to explore the possibility of discovering new loss function formulations via directly searching the primitive operation combinations. So that the learned losses not only fit for diverse object detection challenges to alleviate huge human efforts, but also have better alignment with evaluation metric and good mathematical convergence property. Beyond the previous auto-loss works on face recognition and image classification, our work makes the first attempt to discover new loss functions for the challenging object detection from primitive operation levels. We propose an effective convergence-simulation driven evolutionary search algorithm, called CSE-Autoloss, for speeding up the search progress by regularizing the mathematical rationality of loss candidates via convergence property verification and model optimization simulation. CSE-Autoloss involves the search space that cover a wide range of the possible variants of existing losses and discovers best-searched loss function combination within a short time (around 1.5 wall-clock days). We conduct extensive evaluations of loss function search on popular detectors and validate the good generalization capability of searched losses across diverse architectures and datasets. Our experiments show that the best-discovered loss function combinations outperform default combinations by 1.1% and 0.8% in terms of mAP for two-stage and one-stage detectors on COCO respectively. Our searched losses are available at https://github.com/PerdonLiu/CSE-Autoloss.