# AnyStar: Domain randomized universal star-convex 3D instance segmentation

arXiv: https://arxiv.org/abs/2307.07044 · HTML: https://mlsanity.com/paper/2307.07044/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 62 references that are in the map.
- First posted on arXiv 13 Jul 2023.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining](https://mlsanity.com/paper/2107.09559/) (2021)
- [Segment Anything in Medical Images](https://mlsanity.com/paper/2304.12306/) (2023)
- [Cell Detection with Star-convex Polygons](https://mlsanity.com/paper/1806.03535/) (2018)
- [Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging](https://mlsanity.com/paper/2304.04155/) (2023)
- [SynthMorph: learning contrast-invariant registration without acquired images](https://mlsanity.com/paper/2004.10282/) (2020)
- [Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation](https://mlsanity.com/paper/2002.02255/) (2020)
- [UniverSeg: Universal Medical Image Segmentation](https://mlsanity.com/paper/2304.06131/) (2023)
- [DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation](https://mlsanity.com/paper/1604.02677/) (2016)
- [SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truth](https://mlsanity.com/paper/1810.06498/) (2018)
- [Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images](https://mlsanity.com/paper/1810.00236/) (2018)
- [Robust machine learning segmentation for large-scale analysis of heterogeneous clinical brain MRI datasets](https://mlsanity.com/paper/2209.02032/) (2022)
- [Star-convex Polyhedra for 3D Object Detection and Segmentation in Microscopy](https://mlsanity.com/paper/1908.03636/) (2019)
- [Unsupervised Instance Segmentation in Microscopy Images via Panoptic Domain Adaptation and Task Re-weighting](https://mlsanity.com/paper/2005.02066/) (2020)
- [Data Efficient Unsupervised Domain Adaptation for Cross-Modality Image Segmentation](https://mlsanity.com/paper/1907.02766/) (2019)
- [Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization](https://mlsanity.com/paper/2102.06315/) (2021)
- [Three Dimensional Fluorescence Microscopy Image Synthesis and Segmentation](https://mlsanity.com/paper/1801.07198/) (2018)
- [Contour Proposal Networks for Biomedical Instance Segmentation](https://mlsanity.com/paper/2104.03393/) (2021)
- [Denoising Diffusion Probabilistic Models for Generation of Realistic Fully-Annotated Microscopy Image Data Sets](https://mlsanity.com/paper/2301.10227/) (2023)
- [NucMM Dataset: 3D Neuronal Nuclei Instance Segmentation at Sub-Cubic Millimeter Scale](https://mlsanity.com/paper/2107.05840/) (2021)
- [Deep Learning Based Instance Segmentation in 3D Biomedical Images Using Weak Annotation](https://mlsanity.com/paper/1806.11137/) (2018)
- [Multiclass Weighted Loss for Instance Segmentation of Cluttered Cells](https://mlsanity.com/paper/1802.07465/) (2018)
- [3D fluorescence microscopy data synthesis for segmentation and benchmarking](https://mlsanity.com/paper/2107.10180/) (2021)
- [An Auxiliary Task for Learning Nuclei Segmentation in 3D Microscopy Images](https://mlsanity.com/paper/2002.02857/) (2020)
- [Automatic Segmentation of the Placenta in BOLD MRI Time Series](https://mlsanity.com/paper/2208.02895/) (2022)

## Built on by (most cited first, 15 of 15)

- [Learning General-Purpose Biomedical Volume Representations using Randomized Synthesis](https://mlsanity.com/paper/2411.02372/) (2024)
- [vesselFM: A Foundation Model for Universal 3D Blood Vessel Segmentation](https://mlsanity.com/paper/2411.17386/) (2024)
- [Towards a general-purpose foundation model for fMRI analysis](https://mlsanity.com/paper/2506.11167/) (2025)
- [Domain-randomized deep learning for neuroimage analysis](https://mlsanity.com/paper/2507.13458/) (2025)
- [Neurovascular Segmentation in sOCT with Deep Learning and Synthetic Training Data](https://mlsanity.com/paper/2407.01419/) (2024)
- [Defining the boundaries: challenges and advances in identifying cells in microscopy images](https://mlsanity.com/paper/2311.08269/) (2023)
- [AnyCXR: Human Anatomy Segmentation of Chest X-ray at Any Acquisition Position using Multi-stage Domain Randomized Synthetic Data with Imperfect Annotations and Conditional Joint Annotation Regularization Learning](https://mlsanity.com/paper/2512.17263/) (2025)
- [SAM3D: Zero-Shot Semi-Automatic Segmentation in 3D Medical Images with the Segment Anything Model](https://mlsanity.com/paper/2405.06786/) (2024)
- [A label-free and data-free training strategy for vasculature segmentation in serial sectioning OCT data](https://mlsanity.com/paper/2405.13757/) (2024)
- [Microscopy Image Segmentation via Point and Shape Regularized Data Synthesis](https://mlsanity.com/paper/2308.09835/) (2023)
- [Shape-aware Segmentation of the Placenta in BOLD Fetal MRI Time Series](https://mlsanity.com/paper/2312.05148/) (2023)
- [Search Wide, Focus Deep: Automated Fetal Brain Extraction with Sparse Training Data](https://mlsanity.com/paper/2410.20532/) (2024)
- [On Inverse Problems, Parameter Estimation, and Domain Generalization](https://mlsanity.com/paper/2506.06024/) (2025)
- [Why Invariance is Not Enough for Biomedical Domain Generalization and How to Fix It](https://mlsanity.com/paper/2604.02564/) (2026)
- [Temporal Diffusion Bridge Model for Vessel Extraction Using Physics-informed Synthetic Data](https://mlsanity.com/paper/2610.14105/) (2026)

## Abstract (arXiv)

Star-convex shapes arise across bio-microscopy and radiology in the form of nuclei, nodules, metastases, and other units. Existing instance segmentation networks for such structures train on densely labeled instances for each dataset, which requires substantial and often impractical manual annotation effort. Further, significant reengineering or finetuning is needed when presented with new datasets and imaging modalities due to changes in contrast, shape, orientation, resolution, and density. We present AnyStar, a domain-randomized generative model that simulates synthetic training data of blob-like objects with randomized appearance, environments, and imaging physics to train general-purpose star-convex instance segmentation networks. As a result, networks trained using our generative model do not require annotated images from unseen datasets. A single network trained on our synthesized data accurately 3D segments C. elegans and P. dumerilii nuclei in fluorescence microscopy, mouse cortical nuclei in micro-CT, zebrafish brain nuclei in EM, and placental cotyledons in human fetal MRI, all without any retraining, finetuning, transfer learning, or domain adaptation. Code is available at https://github.com/neel-dey/AnyStar.