# Revealing the Evolution of Order in Materials Microstructures Using Multi-Modal Computer Vision

arXiv: https://arxiv.org/abs/2411.09896 · HTML: https://mlsanity.com/paper/2411.09896/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 87 references that are in the map.
- First posted on arXiv 15 Nov 2024.

## Builds on (most cited first, 17 of 17)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation](https://mlsanity.com/paper/1902.04049/) (2019)
- [Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cut](https://mlsanity.com/paper/2202.11539/) (2022)
- [Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization](https://mlsanity.com/paper/2205.07839/) (2022)
- [Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review](https://mlsanity.com/paper/2004.05224/) (2020)
- [Multimodal learning with graphs](https://mlsanity.com/paper/2209.03299/) (2022)
- [DeepCut: Unsupervised Segmentation using Graph Neural Networks Clustering](https://mlsanity.com/paper/2212.05853/) (2022)
- [AtomAI: A Deep Learning Framework for Analysis of Image and Spectroscopy Data in (Scanning) Transmission Electron Microscopy and Beyond](https://mlsanity.com/paper/2105.07485/) (2021)
- [Machine Learning Pipeline for Segmentation and Defect Identification from High Resolution Transmission Electron Microscopy Data](https://mlsanity.com/paper/2001.05022/) (2020)
- [Deep Learning Segmentation of Complex Features in Atomic-Resolution Phase Contrast Transmission Electron Microscopy Images](https://mlsanity.com/paper/2012.05322/) (2020)
- [Understanding the Influence of Receptive Field and Network Complexity in Neural-Network-Guided TEM Image Analysis](https://mlsanity.com/paper/2204.04250/) (2022)
- [An Automated Scanning Transmission Electron Microscope Guided by Sparse Data Analytics](https://mlsanity.com/paper/2109.14772/) (2021)
- [A Comprehensive and Versatile Multimodal Deep Learning Approach for Predicting Diverse Properties of Advanced Materials](https://mlsanity.com/paper/2303.16412/) (2023)
- [The landscape of ontologies in materials science and engineering: A survey and evaluation](https://mlsanity.com/paper/2408.06034/) (2024)
- [STEM image analysis based on deep learning: identification of vacancy defects and polymorphs of MoS2](https://mlsanity.com/paper/2206.04272/) (2022)
- [Unsupervised segmentation of irradiationunicodex2010induced orderunicodex2010disorder phase transitions in electron microscopy](https://mlsanity.com/paper/2311.08585/) (2023)

## Built on by (most cited first, 1 of 1)

- [Revealing the Hidden Third Dimension of Point Defects in Two-Dimensional MXenes](https://mlsanity.com/paper/2511.08350/) (2025)

## Abstract (arXiv)

The development of high-performance materials for microelectronics, energy storage, and extreme environments depends on our ability to describe and direct property-defining microstructural order. Our present understanding is typically derived from laborious manual analysis of imaging and spectroscopy data, which is difficult to scale, challenging to reproduce, and lacks the ability to reveal latent associations needed for mechanistic models. Here, we demonstrate a multi-modal machine learning (ML) approach to describe order from electron microscopy analysis of the complex oxide La1-xSrxFeO3. We construct a hybrid pipeline based on fully and semi-supervised classification, allowing us to evaluate both the characteristics of each data modality and the value each modality adds to the ensemble. We observe distinct differences in the performance of uni- and multi-modal models, from which we draw general lessons in describing crystal order using computer vision.