# Microscopy Image Segmentation via Point and Shape Regularized Data Synthesis

arXiv: https://arxiv.org/abs/2308.09835 · HTML: https://mlsanity.com/paper/2308.09835/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 31 references that are in the map.
- First posted on arXiv 18 Aug 2023.

## Builds on (most cited first, 25 of 25)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images](https://mlsanity.com/paper/1812.06499/) (2018)
- [Self supervised contrastive learning for digital histopathology](https://mlsanity.com/paper/2011.13971/) (2020)
- [You Only Need Adversarial Supervision for Semantic Image Synthesis](https://mlsanity.com/paper/2012.04781/) (2020)
- [Distribution Matching Losses Can Hallucinate Features in Medical Image Translation](https://mlsanity.com/paper/1805.08841/) (2018)
- [CycleGAN, a Master of Steganography](https://mlsanity.com/paper/1712.02950/) (2017)
- [Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images](https://mlsanity.com/paper/1810.00236/) (2018)
- [Weakly Supervised Deep Nuclei Segmentation Using Partial Points Annotation in Histopathology Images](https://mlsanity.com/paper/2007.05448/) (2020)
- [Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization](https://mlsanity.com/paper/2102.06315/) (2021)
- [Three Dimensional Fluorescence Microscopy Image Synthesis and Segmentation](https://mlsanity.com/paper/1801.07198/) (2018)
- [Harmonic Unpaired Image-to-image Translation](https://mlsanity.com/paper/1902.09727/) (2019)
- [AnyStar: Domain randomized universal star-convex 3D instance segmentation](https://mlsanity.com/paper/2307.07044/) (2023)
- [Which Pixel to Annotate: a Label-Efficient Nuclei Segmentation Framework](https://mlsanity.com/paper/2212.10305/) (2022)
- [Unsupervised Histopathology Image Synthesis](https://mlsanity.com/paper/1712.05021/) (2017)
- [Sharp-GAN: Sharpness Loss Regularized GAN for Histopathology Image Synthesis](https://mlsanity.com/paper/2110.14709/) (2021)
- [Weakly Supervised Nuclei Segmentation via Instance Learning](https://mlsanity.com/paper/2202.01564/) (2022)
- [SIAN: Style-Guided Instance-Adaptive Normalization for Multi-Organ Histopathology Image Synthesis](https://mlsanity.com/paper/2209.02412/) (2022)
- [Point-supervised Segmentation of Microscopy Images and Volumes via Objectness Regularization](https://mlsanity.com/paper/2103.05617/) (2021)
- [ASIST: Annotation-free Synthetic Instance Segmentation and Tracking by Adversarial Simulations](https://mlsanity.com/paper/2101.00567/) (2021)
- [Enhanced Sharp-GAN For Histopathology Image Synthesis](https://mlsanity.com/paper/2301.10187/) (2023)
- [Unsupervised Dense Nuclei Detection and Segmentation with Prior Self-activation Map For Histology Images](https://mlsanity.com/paper/2210.07862/) (2022)

## Abstract (arXiv)

Current deep learning-based approaches for the segmentation of microscopy images heavily rely on large amount of training data with dense annotation, which is highly costly and laborious in practice. Compared to full annotation where the complete contour of objects is depicted, point annotations, specifically object centroids, are much easier to acquire and still provide crucial information about the objects for subsequent segmentation. In this paper, we assume access to point annotations only during training and develop a unified pipeline for microscopy image segmentation using synthetically generated training data. Our framework includes three stages: (1) it takes point annotations and samples a pseudo dense segmentation mask constrained with shape priors; (2) with an image generative model trained in an unpaired manner, it translates the mask to a realistic microscopy image regularized by object level consistency; (3) the pseudo masks along with the synthetic images then constitute a pairwise dataset for training an ad-hoc segmentation model. On the public MoNuSeg dataset, our synthesis pipeline produces more diverse and realistic images than baseline models while maintaining high coherence between input masks and generated images. When using the identical segmentation backbones, the models trained on our synthetic dataset significantly outperform those trained with pseudo-labels or baseline-generated images. Moreover, our framework achieves comparable results to models trained on authentic microscopy images with dense labels, demonstrating its potential as a reliable and highly efficient alternative to labor-intensive manual pixel-wise annotations in microscopy image segmentation. The code is available.