# Improving Combined Detection and Classification of TEM Defects via Mask-Conditioned Latent Diffusion Augmentation

arXiv: https://arxiv.org/abs/2606.02532 · HTML: https://mlsanity.com/paper/2606.02532/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Flow matching, time series and science, one of 14 directions in Diffusion models.
- Builds on 15 of its 44 references that are in the map.
- First posted on arXiv 1 Jun 2026.

## Placed in

- Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/)

## Builds on (most cited first, 15 of 15)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications](https://mlsanity.com/paper/2001.06937/) (2020)
- [Overview: Computer vision and machine learning for microstructural characterization and analysis](https://mlsanity.com/paper/2005.14260/) (2020)
- [Few-shot Image Generation with Diffusion Models](https://mlsanity.com/paper/2211.03264/) (2022)
- [Mask-conditioned latent diffusion for generating gastrointestinal polyp images](https://mlsanity.com/paper/2304.05233/) (2023)
- [A Deep Learning Based Automatic Defect Analysis Framework for In-situ TEM Ion Irradiations](https://mlsanity.com/paper/2108.08882/) (2021)
- [Multi defect detection and analysis of electron microscopy images with deep learning](https://mlsanity.com/paper/2108.08883/) (2021)
- [A deep learned nanowire segmentation model using synthetic data augmentation](https://mlsanity.com/paper/2109.04429/) (2021)
- [Accelerating Domain-Aware Electron Microscopy Analysis Using Deep Learning Models with Synthetic Data and Image-Wide Confidence Scoring](https://mlsanity.com/paper/2408.01558/) (2024)

## Abstract (arXiv)

Analyzing microstructural defects in transmission electron microscopy (TEM) images, particularly in irradiated metal alloys, is often limited by the availability of high-quality, labeled data. To address this, we introduce a generative data augmentation approach using a mask-conditioned latent diffusion model (LDM) for synthesizing realistic TEM images with controllable, automatically labeled multi-class defect masks. Without requiring manual annotations for generation, our method enables the creation of synthetic image-mask pairs by sampling distributions learned from experimental masks. These generated data were used to augment small experimental datasets of varying sizes (10, 50, and 100 labeled experimental images) to train a Mask Regional Convolutional Neural Network (R-CNN) model for defect detection and classification. Our results show that generative augmentation yields small overall model performance improvements, with up to a 0.02 gain in the harmonic mean of detection and classification F1 scores. However, we also find that the relative contributions to detection and classification improvement depend on the specific train/test data split. These findings highlight the potential of targeted generative models to enhance deep learning performance in data-scarce microscopy-based image quantification tasks.