Physics Informed Generative AI Enabling Labour Free Segmentation For Microscopy Analysis
Not on the map. Why?
Abstract (arXiv)
Semantic segmentation of microscopy images is a critical task for high-throughput materials characterisation, yet its automation is severely constrained by the prohibitive cost, subjectivity, and scarcity of expert-annotated data. While physics-based simulations offer a scalable alternative to manual labelling, models trained on such data historically fail to generalise due to a significant domain gap, lacking the complex textures, noise patterns, and imaging artefacts inherent to experimental data. This paper introduces a novel framework for labour-free segmentation that successfully bridges this simulation-to-reality gap. Our pipeline leverages phase-field simulations to generate an abundant source of microstructural morphologies with perfect, intrinsically-derived ground-truth masks. We then employ a Cycle-Consistent Generative Adversarial Network (CycleGAN) for unpaired image-to-image translation, transforming the clean simulations into a large-scale dataset of high-fidelity, realistic SEM images. A U-Net model, trained exclusively on this synthetic data, demonstrated remarkable generalisation when deployed on unseen experimental images, achieving a mean Boundary F1-Score of 0.90 and an Intersection over Union (IOU) of 0.88. Comprehensive validation using t-SNE feature-space projection and Shannon entropy analysis confirms that our synthetic images are statistically and featurally indistinguishable from the real data manifold. By completely decoupling model training from manual annotation, our generative framework transforms a data-scarce problem into one of data abundance, providing a robust and fully automated solution to accelerate materials discovery and analysis.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 7 of its 45 references that are in the map. First posted on arXiv 2 Feb 2026.
Builds on
7 of 45 references.
References · 7
- 2020Review: Deep Learning in Electron Microscopycited by 20
- 2017Advanced Steel Microstructural Classification by Deep Learning Methodscited by 15
- 2019Paradigm shift in electron-based crystallography via machine learningcited by 12
- 2022Disentangling multiple scattering with deep learning: application to strain mapping from electron diffraction patternscited by 7
- 2017Image registration of low signal-to-noise cryo-STEM datacited by 5
- 2021Addressing materials' microstructure diversity using transfer learningcited by 4
- 2022Towards Augmented Microscopy with Reinforcement Learning-Enhanced Workflowscited by 2
Built on by
Not cited yet.