# Machine learning of microstructure--property relationships in materials leveraging microstructure representation from foundational vision transformers

arXiv: https://arxiv.org/abs/2501.18637 · HTML: https://mlsanity.com/paper/2501.18637/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 57 references that are in the map.
- First posted on arXiv 28 Jan 2025.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Adaptive Layer Selection for Efficient Vision Transformer Fine-Tuning](https://mlsanity.com/paper/2408.08670/) (2024)

## Built on by (most cited first, 3 of 3)

- [Graph-Structured Hyperdimensional Computing for Data-Efficient and Explainable Process-Structure-Property Prediction](https://mlsanity.com/paper/2605.07999/) (2026)
- [Compressing the Validation Bottleneck: An Agentic Self-Driving Lab for Scientific Discovery](https://mlsanity.com/paper/2607.04508/) (2026)
- [An Automated and Reproducible Workflow for Crack Identification and Damage Assessment of Fusion Materials](https://mlsanity.com/paper/2610.03505/) (2026)

## Abstract (arXiv)

Machine learning of microstructure--property relationships from data is an emerging approach in computational materials science. Most existing machine learning efforts focus on the development of task-specific models for each microstructure--property relationship. We propose utilizing pre-trained foundational vision transformers for the extraction of task-agnostic microstructure features and subsequent light-weight machine learning of a microstructure-dependent property. We demonstrate our approach with pre-trained state-of-the-art vision transformers (CLIP, DINOv2, SAM) in two case studies on machine-learning: (i) elastic modulus of two-phase microstructures based on simulations data; and (ii) Vicker's hardness of Ni-base and Co-base superalloys based on experimental data published in literature. Our results show the potential of foundational vision transformers for robust microstructure representation and efficient machine learning of microstructure--property relationships without the need for expensive task-specific training or fine-tuning of bespoke deep learning models.