# Mask-RCNN and U-net Ensembled for Nuclei Segmentation

arXiv: https://arxiv.org/abs/1901.10170 · HTML: https://mlsanity.com/paper/1901.10170/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jan 2019.

## Built on by (most cited first, 23 of 23)

- [CPP-Net: Context-aware Polygon Proposal Network for Nucleus Segmentation](https://mlsanity.com/paper/2102.06867/) (2021)
- [FEDZIP: A Compression Framework for Communication-Efficient Federated Learning](https://mlsanity.com/paper/2102.01593/) (2021)
- [Modality specific U-Net variants for biomedical image segmentation: A survey](https://mlsanity.com/paper/2107.04537/) (2021)
- [Cognitive Visual Commonsense Reasoning Using Dynamic Working Memory](https://mlsanity.com/paper/2107.01671/) (2021)
- [Interpretable Visual Understanding with Cognitive Attention Network](https://mlsanity.com/paper/2108.02924/) (2021)
- [Multi-scale Regional Attention Deeplab3+: Multiple Myeloma Plasma Cells Segmentation in Microscopic Images](https://mlsanity.com/paper/2105.06238/) (2021)
- [Cyclic Learning: Bridging Image-level Labels and Nuclei Instance Segmentation](https://mlsanity.com/paper/2306.02691/) (2023)
- [ConvNeXt-backbone HoVerNet for nuclei segmentation and classification](https://mlsanity.com/paper/2202.13560/) (2022)
- [Nuclei & Glands Instance Segmentation in Histology Images: A Narrative Review](https://mlsanity.com/paper/2208.12460/) (2022)
- [Machine Learning-based Automatic Graphene Detection with Color Correction for Optical Microscope Images](https://mlsanity.com/paper/2103.13495/) (2021)
- [Automated 3D Tumor Segmentation using Temporal Cubic PatchGAN (TCuP-GAN)](https://mlsanity.com/paper/2311.14148/) (2023)
- [ShapeICP: Iterative Category-level Object Pose and Shape Estimation from Depth](https://mlsanity.com/paper/2408.13147/) (2024)
- [Using spatial-temporal ensembles of convolutional neural networks for lumen segmentation in ureteroscopy](https://mlsanity.com/paper/2104.01985/) (2021)
- [YOLO2U-Net: Detection-Guided 3D Instance Segmentation for Microscopy](https://mlsanity.com/paper/2207.06215/) (2022)
- [AMA-SAM: Adversarial Multi-Domain Alignment of Segment Anything Model for High-Fidelity Histology Nuclei Segmentation](https://mlsanity.com/paper/2503.21695/) (2025)
- [LSP-DETR: Efficient and Scalable Nuclei Segmentation in Whole-Slide Images](https://mlsanity.com/paper/2601.03163/) (2026)
- [Instance segmentation with the number of clusters incorporated in embedding learning](https://mlsanity.com/paper/2103.14869/) (2021)
- [COVID-19 Lung Lesion Segmentation Using a Sparsely Supervised Mask R-CNN on Chest X-rays Automatically Computed from Volumetric CTs](https://mlsanity.com/paper/2105.08147/) (2021)
- [Semantic Dense Reconstruction with Consistent Scene Segments](https://mlsanity.com/paper/2109.14821/) (2021)
- [Attention Mechanism based Cognition-level Scene Understanding](https://mlsanity.com/paper/2204.08027/) (2022)
- [License Plate Recognition Based On Multi-Angle View Model](https://mlsanity.com/paper/2309.12972/) (2023)
- [Multimodal Laryngoscopic Video Analysis for Assisted Diagnosis of Vocal Fold Paralysis](https://mlsanity.com/paper/2409.03597/) (2024)
- [Weak-to-Strong Generalization Enables Fully Automated De Novo Training of Multi-head Mask-RCNN Model for Segmenting Densely Overlapping Cell Nuclei in Multiplex Whole-slice Brain Images](https://mlsanity.com/paper/2512.11722/) (2025)

## Abstract (arXiv)

Nuclei segmentation is both an important and in some ways ideal task for modern computer vision methods, e.g. convolutional neural networks. While recent developments in theory and open-source software have made these tools easier to implement, expert knowledge is still required to choose the right model architecture and training setup. We compare two popular segmentation frameworks, U-Net and Mask-RCNN in the nuclei segmentation task and find that they have different strengths and failures. To get the best of both worlds, we develop an ensemble model to combine their predictions that can outperform both models by a significant margin and should be considered when aiming for best nuclei segmentation performance.