Multimodal Foundation Models for Material Property Prediction and Discovery
Not on the map. Why?
Abstract (arXiv)
Artificial intelligence is transforming computational materials science, improving the prediction of material properties, and accelerating the discovery of novel materials. Recently, publicly available material data repositories have grown rapidly. This growth encompasses not only more materials but also a greater variety and quantity of their associated properties. Existing machine learning efforts in materials science focus primarily on single-modality tasks, i.e. relationships between materials and a single physical property, thus not taking advantage of the rich and multimodal set of material properties. Here, we introduce Multimodal Learning for Materials (MultiMat), which enables self-supervised multi-modality training of foundation models for materials. We demonstrate our framework's potential using data from the Materials Project database on multiple axes: (i) MultiMat achieves state-of-the-art performance for challenging material property prediction tasks; (ii) MultiMat enables novel and accurate material discovery via latent space similarity, enabling screening for stable materials with desired properties; and (iii) MultiMat encodes interpretable emergent features that may provide novel scientific insights.
Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 32 of its 86 references that are in the map. First posted on arXiv 30 Nov 2023.
Builds on
32 of 86 references.
References · 32
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023GPT-4 Technical Reportcited by 18,132
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
- 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
20 more
- 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
- 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
- 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
- 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
- 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
- 2021Florence: A New Foundation Model for Computer Visioncited by 802
- 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
- 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
- 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 335
- 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 262
- 2022Periodic Graph Transformers for Crystal Material Property Predictioncited by 87
- 2023Identifiability Results for Multimodal Contrastive Learningcited by 58
- 2023Efficient Approximations of Complete Interatomic Potentials for Crystal Property Predictioncited by 37
- 2023LLM-Prop: Predicting Physical And Electronic Properties Of Crystalline Solids From Their Text Descriptionscited by 29
- 2021Interpretable and Explainable Machine Learning for Materials Science and Chemistrycited by 26
- 2022VoLTA: Vision-Language Transformer with Weakly-Supervised Local-Feature Alignmentcited by 20
- 2022Interpretable models for extrapolation in scientific machine learningcited by 11
- 2022Topogivity: A Machine-Learned Chemical Rule for Discovering Topological Materialscited by 7
- 2023Density of States Prediction of Crystalline Materials via Prompt-guided Multi-Modal Transformercited by 3
Built on by
Citing papers per year: 2024: 2; 2025: 10; 2026: 7.
19 papers.
Sort:
By area:
- 2025LeMat-GenBench: A Unified Evaluation Framework for Crystal Generative Modelscited by 20
- 2024Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistrycited by 8
- 2025L2M: Mutual Information Scaling Law for Long-Context Language Modelingcited by 8
- 2025A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Toolscited by 7
- 2024UniMat: Unifying Materials Embeddings through Multi-modal Learningcited by 6
- 2025Bridging Text and Crystal Structures: Literature-driven Contrastive Learning for Materials Sciencecited by 3
- 2025CAST: Cross Attention based multimodal fusion of Structure and Text for materials property predictioncited by 2
- 2025Learning simple heuristic rules for classifying materials based on chemical compositioncited by 2
- 2025Perspective on Utilizing Foundation Models for Laboratory Automation in Materials Researchcited by 2
- 2025Sparse, self-organizing ensembles of local kernels detect rare statistical anomaliescited by 1
- 2025MOCLIP: A Foundation Model for Large-Scale Nanophotonic Inverse Designcited by 1
- 2026MEIDNet: Multimodal generative AI framework for inverse materials designcited by 1
7 more
- 2025XxaCT-NN: Structure Agnostic Multimodal Learning for Materials Sciencecited by 0
- 2026Sustainable Materials Discovery in the Era of Artificial Intelligencecited by 0
- 2026SCALAR: Quantifying Structural Hallucination, Consistency, and Reasoning Gaps in Materials Foundation Modelscited by 0
- 2026Learning ORDER-Aware Multimodal Representations for Composite Materials Designcited by 0
- 2026Towards Automated Discovery: A Review of Generative Models, Multimodal Learning and Closed-Loop Workflows in Inverse Materials Designcited by 0
- 2026MatBind: A Shared Embedding Space for Multimodal Materials Characterizationcited by 0
- 2026PhyMo: A Physical-Field Modality for Multimodal AI4Physicscited by 0
- 2026PhyMo: A Physical-Field Modality for Multimodal AI4Physics23 Sep 2026
- 2026MatBind: A Shared Embedding Space for Multimodal Materials Characterization9 Jul 2026
- 2026Towards Automated Discovery: A Review of Generative Models, Multimodal Learning and Closed-Loop Workflows in Inverse Materials Design1 Jun 2026
- 2026SCALAR: Quantifying Structural Hallucination, Consistency, and Reasoning Gaps in Materials Foundation Models29 Jan 2026
- 2026MEIDNet: Multimodal generative AI framework for inverse materials design29 Jan 2026
- 2026Sustainable Materials Discovery in the Era of Artificial Intelligence29 Jan 2026
- 2026Learning ORDER-Aware Multimodal Representations for Composite Materials Design23 Jan 2026
- 2025LeMat-GenBench: A Unified Evaluation Framework for Crystal Generative Models4 Dec 2025
- 2025MOCLIP: A Foundation Model for Large-Scale Nanophotonic Inverse Design24 Nov 2025
- 2025Sparse, self-organizing ensembles of local kernels detect rare statistical anomalies5 Nov 2025
- 2025XxaCT-NN: Structure Agnostic Multimodal Learning for Materials Science27 Jun 2025
- 2025A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Tools25 Jun 2025
7 more
- 2025Perspective on Utilizing Foundation Models for Laboratory Automation in Materials Research14 Jun 2025
- 2025Learning simple heuristic rules for classifying materials based on chemical composition5 May 2025
- 2025L2M: Mutual Information Scaling Law for Long-Context Language Modeling6 Mar 2025
- 2025CAST: Cross Attention based multimodal fusion of Structure and Text for materials property prediction6 Feb 2025
- 2025Bridging Text and Crystal Structures: Literature-driven Contrastive Learning for Materials Science22 Jan 2025
- 2024Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry20 Nov 2024
- 2024UniMat: Unifying Materials Embeddings through Multi-modal Learning13 Nov 2024
- 2025LeMat-GenBench: A Unified Evaluation Framework for Crystal Generative Modelscited by 20
- 2024Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistrycited by 8
- 2025L2M: Mutual Information Scaling Law for Long-Context Language Modelingcited by 8
- 2024UniMat: Unifying Materials Embeddings through Multi-modal Learningcited by 6
- 2025Bridging Text and Crystal Structures: Literature-driven Contrastive Learning for Materials Sciencecited by 3
- 2025CAST: Cross Attention based multimodal fusion of Structure and Text for materials property predictioncited by 2
- 2025Learning simple heuristic rules for classifying materials based on chemical compositioncited by 2
- 2025Perspective on Utilizing Foundation Models for Laboratory Automation in Materials Researchcited by 2
- 2025Sparse, self-organizing ensembles of local kernels detect rare statistical anomaliescited by 1
- 2025MOCLIP: A Foundation Model for Large-Scale Nanophotonic Inverse Designcited by 1
- 2026MEIDNet: Multimodal generative AI framework for inverse materials designcited by 1
- 2025XxaCT-NN: Structure Agnostic Multimodal Learning for Materials Sciencecited by 0
12 most cited of 17 All 17 (JSON)
- 2026PhyMo: A Physical-Field Modality for Multimodal AI4Physics23 Sep 2026
- 2026MatBind: A Shared Embedding Space for Multimodal Materials Characterization9 Jul 2026
- 2026Towards Automated Discovery: A Review of Generative Models, Multimodal Learning and Closed-Loop Workflows in Inverse Materials Design1 Jun 2026
- 2026MEIDNet: Multimodal generative AI framework for inverse materials design29 Jan 2026
- 2026Sustainable Materials Discovery in the Era of Artificial Intelligence29 Jan 2026
- 2026Learning ORDER-Aware Multimodal Representations for Composite Materials Design23 Jan 2026
- 2025LeMat-GenBench: A Unified Evaluation Framework for Crystal Generative Models4 Dec 2025
- 2025MOCLIP: A Foundation Model for Large-Scale Nanophotonic Inverse Design24 Nov 2025
- 2025Sparse, self-organizing ensembles of local kernels detect rare statistical anomalies5 Nov 2025
- 2025XxaCT-NN: Structure Agnostic Multimodal Learning for Materials Science27 Jun 2025
- 2025Perspective on Utilizing Foundation Models for Laboratory Automation in Materials Research14 Jun 2025
- 2025Learning simple heuristic rules for classifying materials based on chemical composition5 May 2025
12 newest of 17 All 17 (JSON)