# SortedAP: Rethinking evaluation metrics for instance segmentation

arXiv: https://arxiv.org/abs/2309.04887 · HTML: https://mlsanity.com/paper/2309.04887/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 19 references that are in the map.
- First posted on arXiv 9 Sep 2023.

## Builds on (most cited first, 8 of 8)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [TensorMask: A Foundation for Dense Object Segmentation](https://mlsanity.com/paper/1903.12174/) (2019)
- [Localization Recall Precision (LRP): A New Performance Metric for Object Detection](https://mlsanity.com/paper/1807.01696/) (2018)
- [Beyond mAP: Towards better evaluation of instance segmentation](https://mlsanity.com/paper/2207.01614/) (2022)
- [Instance Segmentation of Dense and Overlapping Objects via Layering](https://mlsanity.com/paper/2210.03551/) (2022)
- [High-throughput Phenotyping of Nematode Cysts](https://mlsanity.com/paper/2110.07057/) (2021)

## Built on by (most cited first, 8 of 8)

- [Panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps](https://mlsanity.com/paper/2312.02608/) (2023)
- [Enforcing View-Consistency in Class-Agnostic 3D Segmentation Fields](https://mlsanity.com/paper/2408.09928/) (2024)
- [Single-shot Star-convex Polygon-based Instance Segmentation for Spatially-correlated Biomedical Objects](https://mlsanity.com/paper/2504.12078/) (2025)
- [SoftPQ: Robust Instance Segmentation Evaluation via Soft Matching and Tunable Thresholds](https://mlsanity.com/paper/2505.12155/) (2025)
- [Redefining Instance Matching: A Unified Framework for Part-Aware Matching in Panoptic Segmentation Evaluation](https://mlsanity.com/paper/2605.31094/) (2026)
- [Unsupervised Region-Growing Network for Object Segmentation in Atmospheric Turbulence](https://mlsanity.com/paper/2311.03572/) (2023)
- [CHOTA: A Higher Order Accuracy Metric for Cell Tracking](https://mlsanity.com/paper/2408.11571/) (2024)
- [Maximum Matching Accuracy: An Instance Segmentation Evaluation Metric Utilizing Globally Optimal Matching](https://mlsanity.com/paper/2606.10107/) (2026)

## Abstract (arXiv)

Designing metrics for evaluating instance segmentation revolves around comprehensively considering object detection and segmentation accuracy. However, other important properties, such as sensitivity, continuity, and equality, are overlooked in the current study. In this paper, we reveal that most existing metrics have a limited resolution of segmentation quality. They are only conditionally sensitive to the change of masks or false predictions. For certain metrics, the score can change drastically in a narrow range which could provide a misleading indication of the quality gap between results. Therefore, we propose a new metric called sortedAP, which strictly decreases with both object- and pixel-level imperfections and has an uninterrupted penalization scale over the entire domain. We provide the evaluation toolkit and experiment code at https://www.github.com/looooongChen/sortedAP.