# H-EMD: A Hierarchical Earth Mover's Distance Method for Instance Segmentation

arXiv: https://arxiv.org/abs/2206.01309 · HTML: https://mlsanity.com/paper/2206.01309/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 60 references that are in the map.
- First posted on arXiv 2 Jun 2022.

## Builds on (most cited first, 22 of 22)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images](https://mlsanity.com/paper/1812.06499/) (2018)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [Cell Detection with Star-convex Polygons](https://mlsanity.com/paper/1806.03535/) (2018)
- [FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentation](https://mlsanity.com/paper/1902.09513/) (2019)
- [MILD-Net: Minimal Information Loss Dilated Network for Gland Instance Segmentation in Colon Histology Images](https://mlsanity.com/paper/1806.01963/) (2018)
- [DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation](https://mlsanity.com/paper/1604.02677/) (2016)
- [Deep Watershed Transform for Instance Segmentation](https://mlsanity.com/paper/1611.08303/) (2016)
- [CIA-Net: Robust Nuclei Instance Segmentation with Contour-aware Information Aggregation](https://mlsanity.com/paper/1903.05358/) (2019)
- [TernausNetV2: Fully Convolutional Network for Instance Segmentation](https://mlsanity.com/paper/1806.00844/) (2018)
- [Recurrent Instance Segmentation](https://mlsanity.com/paper/1511.08250/) (2015)
- [Microscopy Cell Segmentation via Convolutional LSTM Networks](https://mlsanity.com/paper/1805.11247/) (2018)
- [Instance Segmentation and Tracking with Cosine Embeddings and Recurrent Hourglass Networks](https://mlsanity.com/paper/1806.02070/) (2018)
- [Instance Segmentation of Biological Images Using Harmonic Embeddings](https://mlsanity.com/paper/1904.05257/) (2019)
- [Instance Segmentation of Biomedical Images with an Object-aware Embedding Learned with Local Constraints](https://mlsanity.com/paper/2004.09821/) (2020)
- [CNN-based Preprocessing to Optimize Watershed-based Cell Segmentation in 3D Confocal Microscopy Images](https://mlsanity.com/paper/1810.06933/) (2018)
- [J Regularization Improves Imbalanced Multiclass Segmentation](https://mlsanity.com/paper/1910.09783/) (2019)
- [Cascade Decoder: A Universal Decoding Method for Biomedical Image Segmentation](https://mlsanity.com/paper/1901.04949/) (2019)
- [SPDA: Superpixel-based Data Augmentation for Biomedical Image Segmentation](https://mlsanity.com/paper/1903.00035/) (2019)
- [The Candidate Multi-Cut for Cell Segmentation](https://mlsanity.com/paper/1707.00907/) (2017)

## Built on by (most cited first, 2 of 2)

- [Cell Instance Segmentation: The Devil Is in the Boundaries](https://mlsanity.com/paper/2510.09848/) (2025)
- [PhagoStat a scalable and interpretable end to end framework for efficient quantification of cell phagocytosis in neurodegenerative disease studies](https://mlsanity.com/paper/2304.13764/) (2023)

## Abstract (arXiv)

Deep learning (DL) based semantic segmentation methods have achieved excellent performance in biomedical image segmentation, producing high quality probability maps to allow extraction of rich instance information to facilitate good instance segmentation. While numerous efforts were put into developing new DL semantic segmentation models, less attention was paid to a key issue of how to effectively explore their probability maps to attain the best possible instance segmentation. We observe that probability maps by DL semantic segmentation models can be used to generate many possible instance candidates, and accurate instance segmentation can be achieved by selecting from them a set of "optimized" candidates as output instances. Further, the generated instance candidates form a well-behaved hierarchical structure (a forest), which allows selecting instances in an optimized manner. Hence, we propose a novel framework, called hierarchical earth mover's distance (H-EMD), for instance segmentation in biomedical 2D+time videos and 3D images, which judiciously incorporates consistent instance selection with semantic-segmentation-generated probability maps. H-EMD contains two main stages. (1) Instance candidate generation: capturing instance-structured information in probability maps by generating many instance candidates in a forest structure. (2) Instance candidate selection: selecting instances from the candidate set for final instance segmentation. We formulate a key instance selection problem on the instance candidate forest as an optimization problem based on the earth mover's distance (EMD), and solve it by integer linear programming. Extensive experiments on eight biomedical video or 3D datasets demonstrate that H-EMD consistently boosts DL semantic segmentation models and is highly competitive with state-of-the-art methods.