# End-to-end training of object class detectors for mean average precision

arXiv: https://arxiv.org/abs/1607.03476 · HTML: https://mlsanity.com/paper/1607.03476/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jul 2016.

## Built on by (most cited first, 30 of 35)

- [LLM4Drive: A Survey of Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.01043/) (2023)
- [GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.17202/) (2021)
- [A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineering](https://mlsanity.com/paper/2306.06211/) (2023)
- [Tuning computer vision models with task rewards](https://mlsanity.com/paper/2302.08242/) (2023)
- [Stochastic Optimization of Areas Under Precision-Recall Curves with Provable Convergence](https://mlsanity.com/paper/2104.08736/) (2021)
- [Momentum Accelerates the Convergence of Stochastic AUPRC Maximization](https://mlsanity.com/paper/2107.01173/) (2021)
- [Cognitive Visual Commonsense Reasoning Using Dynamic Working Memory](https://mlsanity.com/paper/2107.01671/) (2021)
- [Improving Interpretation Faithfulness for Vision Transformers](https://mlsanity.com/paper/2311.17983/) (2023)
- [MetricOpt: Learning to Optimize Black-Box Evaluation Metrics](https://mlsanity.com/paper/2104.10631/) (2021)
- [Exploring the Algorithm-Dependent Generalization of AUPRC Optimization with List Stability](https://mlsanity.com/paper/2209.13262/) (2022)
- [Reliable Decision from Multiple Subtasks through Threshold Optimization: Content Moderation in the Wild](https://mlsanity.com/paper/2208.07522/) (2022)
- [AoP-SAM: Automation of Prompts for Efficient Segmentation](https://mlsanity.com/paper/2505.11980/) (2025)
- [Deep AUC Maximization for Medical Image Classification: Challenges and Opportunities](https://mlsanity.com/paper/2111.02400/) (2021)
- [A Parallel Implementation of Computing Mean Average Precision](https://mlsanity.com/paper/2206.09504/) (2022)
- [Image-Guided Outdoor LiDAR Perception Quality Assessment for Autonomous Driving](https://mlsanity.com/paper/2406.17265/) (2024)
- [DropTrack -- automatic droplet tracking using deep learning for microfluidic applications](https://mlsanity.com/paper/2205.02568/) (2022)
- [Automatic Scene Generation: State-of-the-Art Techniques, Models, Datasets, Challenges, and Future Prospects](https://mlsanity.com/paper/2410.01816/) (2024)
- [Searching Parameterized AP Loss for Object Detection](https://mlsanity.com/paper/2112.05138/) (2021)
- [Segmentation-Consistent Probabilistic Lesion Counting](https://mlsanity.com/paper/2204.05276/) (2022)
- [Identification of lung nodules CT scan using YOLOv5 based on convolution neural network](https://mlsanity.com/paper/2301.02166/) (2022)
- [Toward Clinically Trustworthy Deep Learning: Applying Conformal Prediction to Intracranial Hemorrhage Detection](https://mlsanity.com/paper/2401.08058/) (2024)
- [An In-Depth Analysis of Data Reduction Methods for Sustainable Deep Learning](https://mlsanity.com/paper/2403.15150/) (2024)
- [MorphoSeg: An Uncertainty-Aware Deep Learning Method for Biomedical Segmentation of Complex Cellular Morphologies](https://mlsanity.com/paper/2409.17110/) (2024)
- [What Demands Attention in Urban Street Scenes? From Scene Understanding towards Road Safety: A Survey of Vision-driven Datasets and Studies](https://mlsanity.com/paper/2507.06513/) (2025)
- [Identification of Driver Phone Usage Violations via State-of-the-Art Object Detection with Tracking](https://mlsanity.com/paper/2109.02119/) (2021)
- [Attention Mechanism based Cognition-level Scene Understanding](https://mlsanity.com/paper/2204.08027/) (2022)
- [Application of image-to-image translation in improving pedestrian detection](https://mlsanity.com/paper/2209.03625/) (2022)
- [SAFEMYRIDES: Application of Decentralized Control Edge-Computing to Ridesharing Monitoring Services](https://mlsanity.com/paper/2301.00888/) (2023)
- [MagicEye: An Intelligent Wearable Towards Independent Living of Visually Impaired](https://mlsanity.com/paper/2303.13863/) (2023)
- [Enhanced Automotive Object Detection via RGB-D Fusion in a DiffusionDet Framework](https://mlsanity.com/paper/2406.03129/) (2024)

## Abstract (arXiv)

We present a method for training CNN-based object class detectors directly using mean average precision (mAP) as the training loss, in a truly end-to-end fashion that includes non-maximum suppression (NMS) at training time. This contrasts with the traditional approach of training a CNN for a window classification loss, then applying NMS only at test time, when mAP is used as the evaluation metric in place of classification accuracy. However, mAP following NMS forms a piecewise-constant structured loss over thousands of windows, with gradients that do not convey useful information for gradient descent. Hence, we define new, general gradient-like quantities for piecewise constant functions, which have wide applicability. We describe how to calculate these efficiently for mAP following NMS, enabling to train a detector based on Fast R-CNN directly for mAP. This model achieves equivalent performance to the standard Fast R-CNN on the PASCAL VOC 2007 and 2012 datasets, while being conceptually more appealing as the very same model and loss are used at both training and test time.