# SoftPQ: Robust Instance Segmentation Evaluation via Soft Matching and Tunable Thresholds

arXiv: https://arxiv.org/abs/2505.12155 · HTML: https://mlsanity.com/paper/2505.12155/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 14 references that are in the map.
- First posted on arXiv 17 May 2025.

## Builds on (most cited first, 8 of 8)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Metrics reloaded: Recommendations for image analysis validation](https://mlsanity.com/paper/2206.01653/) (2022)
- [Boundary IoU: Improving Object-Centric Image Segmentation Evaluation](https://mlsanity.com/paper/2103.16562/) (2021)
- [Revisiting Evaluation Metrics for Semantic Segmentation: Optimization and Evaluation of Fine-grained Intersection over Union](https://mlsanity.com/paper/2310.19252/) (2023)
- [Boundary Difference Over Union Loss For Medical Image Segmentation](https://mlsanity.com/paper/2308.00220/) (2023)
- [SortedAP: Rethinking evaluation metrics for instance segmentation](https://mlsanity.com/paper/2309.04887/) (2023)
- [Beyond mAP: Towards better evaluation of instance segmentation](https://mlsanity.com/paper/2207.01614/) (2022)

## Built on by (most cited first, 1 of 1)

- [Maximum Matching Accuracy: An Instance Segmentation Evaluation Metric Utilizing Globally Optimal Matching](https://mlsanity.com/paper/2606.10107/) (2026)

## Abstract (arXiv)

Segmentation evaluation metrics traditionally rely on binary decision logic: predictions are either correct or incorrect, based on rigid IoU thresholds. Detection--based metrics such as F1 and mAP determine correctness at the object level using fixed overlap cutoffs, while overlap--based metrics like Intersection over Union (IoU) and Dice operate at the pixel level, often overlooking instance--level structure. Panoptic Quality (PQ) attempts to unify detection and segmentation assessment, but it remains dependent on hard-threshold matching--treating predictions below the threshold as entirely incorrect. This binary framing obscures important distinctions between qualitatively different errors and fails to reward gradual model improvements. We propose SoftPQ, a flexible and interpretable instance segmentation metric that redefines evaluation as a graded continuum rather than a binary classification. SoftPQ introduces tunable upper and lower IoU thresholds to define a partial matching region and applies a sublinear penalty function to ambiguous or fragmented predictions. These extensions allow SoftPQ to exhibit smoother score behavior, greater robustness to structural segmentation errors, and more informative feedback for model development and evaluation. Through controlled perturbation experiments, we show that SoftPQ captures meaningful differences in segmentation quality that existing metrics overlook, making it a practical and principled alternative for both benchmarking and iterative model refinement.