# A robust synthetic data generation framework for machine learning in High-Resolution Transmission Electron Microscopy (HRTEM)

arXiv: https://arxiv.org/abs/2309.06122 · HTML: https://mlsanity.com/paper/2309.06122/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 48 references that are in the map.
- First posted on arXiv 12 Sep 2023.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets](https://mlsanity.com/paper/2201.02177/) (2022)
- [UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation](https://mlsanity.com/paper/1912.05074/) (2019)
- [Learning Local Equivariant Representations for Large-Scale Atomistic Dynamics](https://mlsanity.com/paper/2204.05249/) (2022)
- [How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks](https://mlsanity.com/paper/2009.11848/) (2020)
- [Boundary loss for highly unbalanced segmentation](https://mlsanity.com/paper/1812.07032/) (2018)
- [Understanding the Failure Modes of Out-of-Distribution Generalization](https://mlsanity.com/paper/2010.15775/) (2020)
- [Machine Learning Pipeline for Segmentation and Defect Identification from High Resolution Transmission Electron Microscopy Data](https://mlsanity.com/paper/2001.05022/) (2020)
- [Understanding the Influence of Receptive Field and Network Complexity in Neural-Network-Guided TEM Image Analysis](https://mlsanity.com/paper/2204.04250/) (2022)
- [Developing and Evaluating Deep Neural Network-based Denoising for Nanoparticle TEM Images with Ultra-low Signal-to-Noise](https://mlsanity.com/paper/2101.07770/) (2021)
- [Leveraging generative adversarial networks to create realistic scanning transmission electron microscopy images](https://mlsanity.com/paper/2301.07743/) (2023)
- [Disentangling multiple scattering with deep learning: application to strain mapping from electron diffraction patterns](https://mlsanity.com/paper/2202.00204/) (2022)

## Built on by (most cited first, 3 of 3)

- [SAM-EM: Real-Time Segmentation for Automated Liquid Phase Transmission Electron Microscopy](https://mlsanity.com/paper/2501.03153/) (2025)
- [Contrast transfer functions help quantify neural network out-of-distribution generalization in HRTEM](https://mlsanity.com/paper/2512.09067/) (2025)
- [High-Fidelity Synthetic Transmission Electron Microscopy Image Generation Using Diffusion Probabilistic Models for Data-Limited Semiconductor Metrology](https://mlsanity.com/paper/2606.24817/) (2026)

## Abstract (arXiv)

Machine learning techniques are attractive options for developing highly-accurate automated analysis tools for nanomaterials characterization, including high-resolution transmission electron microscopy (HRTEM). However, successfully implementing such machine learning tools can be difficult due to the challenges in procuring sufficiently large, high-quality training datasets from experiments. In this work, we introduce Construction Zone, a Python package for rapidly generating complex nanoscale atomic structures, and develop an end-to-end workflow for creating large simulated databases for training neural networks. Construction Zone enables fast, systematic sampling of realistic nanomaterial structures, and can be used as a random structure generator for simulated databases, which is important for generating large, diverse synthetic datasets. Using HRTEM imaging as an example, we train a series of neural networks on various subsets of our simulated databases to segment nanoparticles and holistically study the data curation process to understand how various aspects of the curated simulated data -- including simulation fidelity, the distribution of atomic structures, and the distribution of imaging conditions -- affect model performance across several experimental benchmarks. Using our results, we are able to achieve state-of-the-art segmentation performance on experimental HRTEM images of nanoparticles from several experimental benchmarks and, further, we discuss robust strategies for consistently achieving high performance with machine learning in experimental settings using purely synthetic data.