# Overview: Computer vision and machine learning for microstructural characterization and analysis

arXiv: https://arxiv.org/abs/2005.14260 · HTML: https://mlsanity.com/paper/2005.14260/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2020.

## Built on by (most cited first, 22 of 22)

- [Peridynamic Neural Operators: A Data-Driven Nonlocal Constitutive Model for Complex Material Responses](https://mlsanity.com/paper/2401.06070/) (2024)
- [Recent Advances and Applications of Machine Learning in Experimental Solid Mechanics: A Review](https://mlsanity.com/paper/2303.07647/) (2023)
- [Can Multimodal LLMs See Materials Clearly? A Multimodal Benchmark on Materials Characterization](https://mlsanity.com/paper/2509.09307/) (2025)
- [An End-to-End Computer Vision Methodology for Quantitative Metallography](https://mlsanity.com/paper/2104.11159/) (2021)
- [Computer Vision Methods for the Microstructural Analysis of Materials: The State-of-the-art and Future Perspectives](https://mlsanity.com/paper/2208.04149/) (2022)
- [Graph convolutional network for predicting abnormal grain growth in Monte Carlo simulations of microstructural evolution](https://mlsanity.com/paper/2110.09326/) (2021)
- [Digital Fingerprinting of Microstructures](https://mlsanity.com/paper/2203.13718/) (2022)
- [Deep Learning-Driven Prediction of Microstructure Evolution via Latent Space Interpolation](https://mlsanity.com/paper/2508.01822/) (2025)
- [Reward driven discovery of the optimal microstructure representations with invariant variational autoencoders](https://mlsanity.com/paper/2510.00243/) (2025)
- [Context Determines Optimal Architecture in Materials Segmentation](https://mlsanity.com/paper/2602.04154/) (2026)
- [Performance, Successes and Limitations of Deep Learning Semantic Segmentation of Multiple Defects in Transmission Electron Micrographs](https://mlsanity.com/paper/2110.08244/) (2021)
- [Computational Discovery of Energy-Efficient Heat Treatment for Microstructure Design using Deep Reinforcement Learning](https://mlsanity.com/paper/2209.11259/) (2022)
- [Automated Grain Boundary (GB) Segmentation and Microstructural Analysis in 347H Stainless Steel Using Deep Learning and Multimodal Microscopy](https://mlsanity.com/paper/2305.07790/) (2023)
- [Procedural Generation of Grain Orientations using the Wave Function Collapse Algorithm](https://mlsanity.com/paper/2311.12272/) (2023)
- [A versatile machine learning workflow for high-throughput analysis of supported metal catalyst particles](https://mlsanity.com/paper/2410.01213/) (2024)
- [Predicting Performance of Object Detection Models in Electron Microscopy Using Random Forests](https://mlsanity.com/paper/2501.08465/) (2025)
- [3D variational autoencoder for fingerprinting microstructure volume elements](https://mlsanity.com/paper/2503.17427/) (2025)
- [MicroLad: 2D-to-3D Microstructure Reconstruction and Generation via Latent Diffusion and Score Distillation](https://mlsanity.com/paper/2508.20138/) (2025)
- [Improving Combined Detection and Classification of TEM Defects via Mask-Conditioned Latent Diffusion Augmentation](https://mlsanity.com/paper/2606.02532/) (2026)
- [Speeding up the annotation process in semantic segmentation industrial applications](https://mlsanity.com/paper/2606.19934/) (2026)
- [Physics as the label for measuring and correcting materials reasoning in multimodal models](https://mlsanity.com/paper/2609.12181/) (2026)
- [Inferring Dislocation Microstructures from X-ray Diffraction via Cross-Modal Contrastive Learning](https://mlsanity.com/paper/2609.12713/) (2026)

## Abstract (arXiv)

The characterization and analysis of microstructure is the foundation of microstructural science, connecting the materials structure to its composition, process history, and properties. Microstructural quantification traditionally involves a human deciding a priori what to measure and then devising a purpose-built method for doing so. However, recent advances in data science, including computer vision (CV) and machine learning (ML) offer new approaches to extracting information from microstructural images. This overview surveys CV approaches to numerically encode the visual information contained in a microstructural image, which then provides input to supervised or unsupervised ML algorithms that find associations and trends in the high-dimensional image representation. CV/ML systems for microstructural characterization and analysis span the taxonomy of image analysis tasks, including image classification, semantic segmentation, object detection, and instance segmentation. These tools enable new approaches to microstructural analysis, including the development of new, rich visual metrics and the discovery of processing-microstructure-property relationships.