# A Deep Learning Based Automatic Defect Analysis Framework for In-situ TEM Ion Irradiations

arXiv: https://arxiv.org/abs/2108.08882 · HTML: https://mlsanity.com/paper/2108.08882/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 73 references that are in the map.
- First posted on arXiv 19 Aug 2021.

## Builds on (most cited first, 14 of 14)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [You Only Look Twice: Rapid Multi-Scale Object Detection In Satellite Imagery](https://mlsanity.com/paper/1805.09512/) (2018)
- [SlimYOLOv3: Narrower, Faster and Better for Real-Time UAV Applications](https://mlsanity.com/paper/1907.11093/) (2019)
- [Assessing Graph-based Deep Learning Models for Predicting Flash Point](https://mlsanity.com/paper/2002.11315/) (2020)
- [Automated Classification of Helium Ingress in Irradiated X-750](https://mlsanity.com/paper/1912.04252/) (2019)
- [MLography: An Automated Quantitative Metallography Model for Impurities Anomaly Detection using Novel Data Mining and Deep Learning Approach](https://mlsanity.com/paper/2003.04226/) (2020)
- [Microscope 2.0: An Augmented Reality Microscope with Real-time Artificial Intelligence Integration](https://mlsanity.com/paper/1812.00825/) (2018)

## Built on by (most cited first, 10 of 10)

- [Performance, Successes and Limitations of Deep Learning Semantic Segmentation of Multiple Defects in Transmission Electron Micrographs](https://mlsanity.com/paper/2110.08244/) (2021)
- [Machine learning for classifying and interpreting coherent X-ray speckle patterns](https://mlsanity.com/paper/2211.08194/) (2022)
- [An Efficient Instance Segmentation Approach for Extracting Fission Gas Bubbles on U-10Zr Annular Fuel](https://mlsanity.com/paper/2302.12833/) (2023)
- [Linking the Dynamic PicoProbe Analytical Electron-Optical Beam Line / Microscope to Supercomputers](https://mlsanity.com/paper/2308.13701/) (2023)
- [Accelerating Ensemble Error Bar Prediction with Single Models Fits](https://mlsanity.com/paper/2404.09896/) (2024)
- [Predicting Performance of Object Detection Models in Electron Microscopy Using Random Forests](https://mlsanity.com/paper/2501.08465/) (2025)
- [Improving U-Net Confidence on TEM Image Data with L2-Regularization, Transfer Learning, and Deep Fine-Tuning](https://mlsanity.com/paper/2507.16779/) (2025)
- [Novelty-Driven Target-Space Discovery in Automated Electron and Scanning Probe Microscopy](https://mlsanity.com/paper/2603.16715/) (2026)
- [AI-assisted Human-in-the-Loop Web Platform for Structural Characterization in Hard drive design](https://mlsanity.com/paper/2604.00359/) (2026)
- [Improving Combined Detection and Classification of TEM Defects via Mask-Conditioned Latent Diffusion Augmentation](https://mlsanity.com/paper/2606.02532/) (2026)

## Abstract (arXiv)

Videos captured using Transmission Electron Microscopy (TEM) can encode details regarding the morphological and temporal evolution of a material by taking snapshots of the microstructure sequentially. However, manual analysis of such video is tedious, error-prone, unreliable, and prohibitively time-consuming if one wishes to analyze a significant fraction of frames for even videos of modest length. In this work, we developed an automated TEM video analysis system for microstructural features based on the advanced object detection model called YOLO and tested the system on an in-situ ion irradiation TEM video of dislocation loops formed in a FeCrAl alloy. The system provides analysis of features observed in TEM including both static and dynamic properties using the YOLO-based defect detection module coupled to a geometry analysis module and a dynamic tracking module. Results show that the system can achieve human comparable performance with an F1 score of 0.89 for fast, consistent, and scalable frame-level defect analysis. This result is obtained on a real but exceptionally clean and stable data set and more challenging data sets may not achieve this performance. The dynamic tracking also enabled evaluation of individual defect evolution like per defect growth rate at a fidelity never before achieved using common human analysis methods. Our work shows that automatically detecting and tracking interesting microstructures and properties contained in TEM videos is viable and opens new doors for evaluating materials dynamics.