# Boundary IoU: Improving Object-Centric Image Segmentation Evaluation

arXiv: https://arxiv.org/abs/2103.16562 · HTML: https://mlsanity.com/paper/2103.16562/

## Key facts

- Cited by 188 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 41 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [Soft-NMS -- Improving Object Detection With One Line of Code](https://mlsanity.com/paper/1704.04503/) (2017)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [Mask Scoring R-CNN](https://mlsanity.com/paper/1903.00241/) (2019)
- [UPSNet: A Unified Panoptic Segmentation Network](https://mlsanity.com/paper/1901.03784/) (2019)
- [Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild](https://mlsanity.com/paper/2007.15649/) (2020)
- [Instance-aware Semantic Segmentation via Multi-task Network Cascades](https://mlsanity.com/paper/1512.04412/) (2015)
- [TIDE: A General Toolbox for Identifying Object Detection Errors](https://mlsanity.com/paper/2008.08115/) (2020)
- [PolyTransform: Deep Polygon Transformer for Instance Segmentation](https://mlsanity.com/paper/1912.02801/) (2019)
- [Synthetic Training for Accurate 3D Human Pose and Shape Estimation in the Wild](https://mlsanity.com/paper/2009.10013/) (2020)
- [Boundary-preserving Mask R-CNN](https://mlsanity.com/paper/2007.08921/) (2020)
- [Efficient Segmentation: Learning Downsampling Near Semantic Boundaries](https://mlsanity.com/paper/1907.07156/) (2019)
- [Joint COCO and Mapillary Workshop at ICCV 2019: COCO Instance Segmentation Challenge Track](https://mlsanity.com/paper/2010.02475/) (2020)
- [Compositional Scalable Object SLAM](https://mlsanity.com/paper/2011.02658/) (2020)

## Built on by (most cited first, 30 of 188)

- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [GroupViT: Semantic Segmentation Emerges from Text Supervision](https://mlsanity.com/paper/2202.11094/) (2022)
- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [Personalize Segment Anything Model with One Shot](https://mlsanity.com/paper/2305.03048/) (2023)
- [Metrics reloaded: Recommendations for image analysis validation](https://mlsanity.com/paper/2206.01653/) (2022)
- [Disentangled Representation Learning](https://mlsanity.com/paper/2211.11695/) (2022)
- [Highly Accurate Dichotomous Image Segmentation](https://mlsanity.com/paper/2203.03041/) (2022)
- [Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformers](https://mlsanity.com/paper/2109.03814/) (2021)
- [Large-scale Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2106.03149/) (2021)
- [SOLQ: Segmenting Objects by Learning Queries](https://mlsanity.com/paper/2106.02351/) (2021)
- [MSeg3D: Multi-modal 3D Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2303.08600/) (2023)
- [Mask Transfiner for High-Quality Instance Segmentation](https://mlsanity.com/paper/2111.13673/) (2021)
- [Common Limitations of Image Processing Metrics: A Picture Story](https://mlsanity.com/paper/2104.05642/) (2021)
- [Understanding metric-related pitfalls in image analysis validation](https://mlsanity.com/paper/2302.01790/) (2023)
- [Learning to Upsample by Learning to Sample](https://mlsanity.com/paper/2308.15085/) (2023)
- [Contrastive Boundary Learning for Point Cloud Segmentation](https://mlsanity.com/paper/2203.05272/) (2022)
- [The TopCoW Challenge -- Topology-Aware Circle of Willis Segmentation for CT and MR Angiography](https://mlsanity.com/paper/2312.17670/) (2023)
- [Eyes Tell All: Irregular Pupil Shapes Reveal GAN-generated Faces](https://mlsanity.com/paper/2109.00162/) (2021)
- [Frequency-aware Feature Fusion for Dense Image Prediction](https://mlsanity.com/paper/2408.12879/) (2024)
- [MobileVLM: A Vision-Language Model for Better Intra- and Inter-UI Understanding](https://mlsanity.com/paper/2409.14818/) (2024)
- [EdgeSAM: Prompt-In-the-Loop Distillation for SAM](https://mlsanity.com/paper/2312.06660/) (2023)
- [On Model Calibration for Long-Tailed Object Detection and Instance Segmentation](https://mlsanity.com/paper/2107.02170/) (2021)
- [Revisiting Image Pyramid Structure for High Resolution Salient Object Detection](https://mlsanity.com/paper/2209.09475/) (2022)
- [GAN-generated Faces Detection: A Survey and New Perspectives](https://mlsanity.com/paper/2202.07145/) (2022)
- [Proposal-Free Temporal Action Detection via Global Segmentation Mask Learning](https://mlsanity.com/paper/2207.06580/) (2022)
- [SegRefiner: Towards Model-Agnostic Segmentation Refinement with Discrete Diffusion Process](https://mlsanity.com/paper/2312.12425/) (2023)
- [Video Mask Transfiner for High-Quality Video Instance Segmentation](https://mlsanity.com/paper/2207.14012/) (2022)
- [E2EC: An End-to-End Contour-based Method for High-Quality High-Speed Instance Segmentation](https://mlsanity.com/paper/2203.04074/) (2022)
- [CAT-SAM: Conditional Tuning for Few-Shot Adaptation of Segment Anything Model](https://mlsanity.com/paper/2402.03631/) (2024)
- [UniVS: Unified and Universal Video Segmentation with Prompts as Queries](https://mlsanity.com/paper/2402.18115/) (2024)

## Abstract (arXiv)

We present Boundary IoU (Intersection-over-Union), a new segmentation evaluation measure focused on boundary quality. We perform an extensive analysis across different error types and object sizes and show that Boundary IoU is significantly more sensitive than the standard Mask IoU measure to boundary errors for large objects and does not over-penalize errors on smaller objects. The new quality measure displays several desirable characteristics like symmetry w.r.t. prediction/ground truth pairs and balanced responsiveness across scales, which makes it more suitable for segmentation evaluation than other boundary-focused measures like Trimap IoU and F-measure. Based on Boundary IoU, we update the standard evaluation protocols for instance and panoptic segmentation tasks by proposing the Boundary AP (Average Precision) and Boundary PQ (Panoptic Quality) metrics, respectively. Our experiments show that the new evaluation metrics track boundary quality improvements that are generally overlooked by current Mask IoU-based evaluation metrics. We hope that the adoption of the new boundary-sensitive evaluation metrics will lead to rapid progress in segmentation methods that improve boundary quality.