# Foundation Model for Composite Microstructures: Reconstruction, Stiffness, and Nonlinear Behavior Prediction

arXiv: https://arxiv.org/abs/2411.06565 · HTML: https://mlsanity.com/paper/2411.06565/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 35 references that are in the map.
- First posted on arXiv 10 Nov 2024.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Structure-based out-of-distribution (OOD) materials property prediction: a benchmark study](https://mlsanity.com/paper/2401.08032/) (2024)
- [Prediction and optimization of mechanical properties of composites using convolutional neural networks](https://mlsanity.com/paper/1906.00094/) (2019)
- [Masked Autoencoders for Unsupervised Anomaly Detection in Medical Images](https://mlsanity.com/paper/2307.07534/) (2023)
- [LS-DYNA Machine Learning-based Multiscale Method for Nonlinear Modeling of Short Fiber-Reinforced Composites](https://mlsanity.com/paper/2301.02738/) (2023)
- [Intelligent multiscale simulation based on process-guided composite database](https://mlsanity.com/paper/2003.09491/) (2020)
- [MAX: Masked Autoencoder for X-ray Fluorescence in Geological Investigation](https://mlsanity.com/paper/2410.12330/) (2024)

## Built on by (most cited first, 2 of 2)

- [Systematic Performance Assessment of Deep Material Networks for Multiscale Material Modeling](https://mlsanity.com/paper/2602.07192/) (2026)
- [Deep material network for homogenization of piezoelectric composites](https://mlsanity.com/paper/2606.22566/) (2026)

## Abstract (arXiv)

We present the Material Masked Autoencoder (MMAE), a self-supervised Vision Transformer pretrained on a large corpus of short-fiber composite images via masked image reconstruction. The pretrained MMAE learns latent representations that capture essential microstructural features and are broadly transferable across tasks. We demonstrate two key applications: (i) predicting homogenized stiffness components through fine-tuning on limited data, and (ii) inferring physically interpretable parameters by coupling MMAE with an interaction-based material network (IMN), thereby enabling extrapolation of nonlinear stress-strain responses. These results highlight the promise of microstructure foundation models and lay the groundwork for future extensions to more complex systems, such as 3D composites and experimental datasets.