# Star-convex Polyhedra for 3D Object Detection and Segmentation in Microscopy

arXiv: https://arxiv.org/abs/1908.03636 · HTML: https://mlsanity.com/paper/1908.03636/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Aug 2019.

## Built on by (most cited first, 30 of 45)

- [Nuclei instance segmentation and classification in histopathology images with StarDist](https://mlsanity.com/paper/2203.02284/) (2022)
- [Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models](https://mlsanity.com/paper/2103.00429/) (2021)
- [A co-evolving agentic AI system for medical imaging analysis](https://mlsanity.com/paper/2509.20279/) (2025)
- [EmbedTrack -- Simultaneous Cell Segmentation and Tracking Through Learning Offsets and Clustering Bandwidths](https://mlsanity.com/paper/2204.10713/) (2022)
- [AnyStar: Domain randomized universal star-convex 3D instance segmentation](https://mlsanity.com/paper/2307.07044/) (2023)
- [Predicting Ki67, ER, PR, and HER2 Statuses from H&E-stained Breast Cancer Images](https://mlsanity.com/paper/2308.01982/) (2023)
- [NucMM Dataset: 3D Neuronal Nuclei Instance Segmentation at Sub-Cubic Millimeter Scale](https://mlsanity.com/paper/2107.05840/) (2021)
- [CellTranspose: Few-shot Domain Adaptation for Cellular Instance Segmentation](https://mlsanity.com/paper/2212.14121/) (2022)
- [Probabilistic Spatial Analysis in Quantitative Microscopy with Uncertainty-Aware Cell Detection using Deep Bayesian Regression of Density Maps](https://mlsanity.com/paper/2102.11865/) (2021)
- [μSplit: efficient image decomposition for microscopy data](https://mlsanity.com/paper/2211.12872/) (2022)
- [Improving 3D deep learning segmentation with biophysically motivated cell synthesis](https://mlsanity.com/paper/2408.16471/) (2024)
- [Tiling artifacts and trade-offs of feature normalization in the segmentation of large biological images](https://mlsanity.com/paper/2503.19545/) (2025)
- [PyTorch Connectomics: A Scalable and Flexible Segmentation Framework for EM Connectomics](https://mlsanity.com/paper/2112.05754/) (2021)
- [Instance Segmentation of Unlabeled Modalities via Cyclic Segmentation GAN](https://mlsanity.com/paper/2204.03082/) (2022)
- [Self-supervised dense representation learning for live-cell microscopy with time arrow prediction](https://mlsanity.com/paper/2305.05511/) (2023)
- [Morphological Profiling for Drug Discovery in the Era of Deep Learning](https://mlsanity.com/paper/2312.07899/) (2023)
- [MMV_Im2Im: An Open Source Microscopy Machine Vision Toolbox for Image-to-Image Transformation](https://mlsanity.com/paper/2209.02498/) (2022)
- [ReStainGAN: Leveraging IHC to IF Stain Domain Translation for in-silico Data Generation](https://mlsanity.com/paper/2403.06545/) (2024)
- [FISBe: A real-world benchmark dataset for instance segmentation of long-range thin filamentous structures](https://mlsanity.com/paper/2404.00130/) (2024)
- [SR-CACO-2: A Dataset for Confocal Fluorescence Microscopy Image Super-Resolution](https://mlsanity.com/paper/2406.09168/) (2024)
- [VISTA-PATH: An interactive foundation model for pathology image segmentation and quantitative analysis in computational pathology](https://mlsanity.com/paper/2601.16451/) (2026)
- [ConsInstancy: Learning Instance Representations for Semi-Supervised Panoptic Segmentation of Concrete Aggregate Particles](https://mlsanity.com/paper/2204.04635/) (2022)
- [YOLO2U-Net: Detection-Guided 3D Instance Segmentation for Microscopy](https://mlsanity.com/paper/2207.06215/) (2022)
- [A Semi-automatic Cell Tracking Process Towards Completing the 4D Atlas of C. elegans Development](https://mlsanity.com/paper/2207.13611/) (2022)
- [Organelle-specific segmentation, spatial analysis, and visualization of volume electron microscopy datasets](https://mlsanity.com/paper/2303.03876/) (2023)
- [Uncertainty Estimation in Instance Segmentation with Star-convex Shapes](https://mlsanity.com/paper/2309.10513/) (2023)
- [TriSAM: Tri-Plane SAM for zero-shot cortical blood vessel segmentation in VEM images](https://mlsanity.com/paper/2401.13961/) (2024)
- [Generalist Segmentation Algorithm for Photoreceptors Analysis in Adaptive Optics Imaging](https://mlsanity.com/paper/2408.14810/) (2024)
- [PNR: Physics-informed Neural Representation for high-resolution LFM reconstruction](https://mlsanity.com/paper/2409.18223/) (2024)
- [Multimodal Alignment of Histopathological Images Using Cell Segmentation and Point Set Matching for Integrative Cancer Analysis](https://mlsanity.com/paper/2410.00152/) (2024)

## Abstract (arXiv)

Accurate detection and segmentation of cell nuclei in volumetric (3D) fluorescence microscopy datasets is an important step in many biomedical research projects. Although many automated methods for these tasks exist, they often struggle for images with low signal-to-noise ratios and/or dense packing of nuclei. It was recently shown for 2D microscopy images that these issues can be alleviated by training a neural network to directly predict a suitable shape representation (star-convex polygon) for cell nuclei. In this paper, we adopt and extend this approach to 3D volumes by using star-convex polyhedra to represent cell nuclei and similar shapes. To that end, we overcome the challenges of 1) finding parameter-efficient star-convex polyhedra representations that can faithfully describe cell nuclei shapes, 2) adapting to anisotropic voxel sizes often found in fluorescence microscopy datasets, and 3) efficiently computing intersections between pairs of star-convex polyhedra (required for non-maximum suppression). Although our approach is quite general, since star-convex polyhedra include common shapes like bounding boxes and spheres as special cases, our focus is on accurate detection and segmentation of cell nuclei. Finally, we demonstrate on two challenging datasets that our approach (StarDist-3D) leads to superior results when compared to classical and deep learning based methods.