mlsanityData to 7 Oct 2026

Multimodal Foundation Models for Material Property Prediction and Discovery

arXiv 2312.00111 ↗First posted 30 Nov 2023cs.LG cond-mat.mtrl-scicited by 19

Not on the map. Why?

Abstract (arXiv)

Artificial intelligence is transforming computational materials science, improving the prediction of material properties, and accelerating the discovery of novel materials. Recently, publicly available material data repositories have grown rapidly. This growth encompasses not only more materials but also a greater variety and quantity of their associated properties. Existing machine learning efforts in materials science focus primarily on single-modality tasks, i.e. relationships between materials and a single physical property, thus not taking advantage of the rich and multimodal set of material properties. Here, we introduce Multimodal Learning for Materials (MultiMat), which enables self-supervised multi-modality training of foundation models for materials. We demonstrate our framework's potential using data from the Materials Project database on multiple axes: (i) MultiMat achieves state-of-the-art performance for challenging material property prediction tasks; (ii) MultiMat enables novel and accurate material discovery via latent space similarity, enabling screening for stable materials with desired properties; and (iii) MultiMat encodes interpretable emergent features that may provide novel scientific insights.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 32 of its 86 references that are in the map. First posted on arXiv 30 Nov 2023.

Builds on

32 of 86 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  10. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  11. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  12. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
20 more
  1. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  2. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  3. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  4. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  5. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  6. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  7. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  8. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
  9. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  10. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 335
  11. 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 262
  12. 2022Periodic Graph Transformers for Crystal Material Property Predictioncited by 87
  13. 2023Identifiability Results for Multimodal Contrastive Learningcited by 58
  14. 2023Efficient Approximations of Complete Interatomic Potentials for Crystal Property Predictioncited by 37
  15. 2023LLM-Prop: Predicting Physical And Electronic Properties Of Crystalline Solids From Their Text Descriptionscited by 29
  16. 2021Interpretable and Explainable Machine Learning for Materials Science and Chemistrycited by 26
  17. 2022VoLTA: Vision-Language Transformer with Weakly-Supervised Local-Feature Alignmentcited by 20
  18. 2022Interpretable models for extrapolation in scientific machine learningcited by 11
  19. 2022Topogivity: A Machine-Learned Chemical Rule for Discovering Topological Materialscited by 7
  20. 2023Density of States Prediction of Crystalline Materials via Prompt-guided Multi-Modal Transformercited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 10; 2026: 7.

19 papers.

Sort:

By area:

  1. 2025LeMat-GenBench: A Unified Evaluation Framework for Crystal Generative Modelscited by 20
  2. 2024Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistrycited by 8
  3. 2025L2M: Mutual Information Scaling Law for Long-Context Language Modelingcited by 8
  4. 2025A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Toolscited by 7
  5. 2024UniMat: Unifying Materials Embeddings through Multi-modal Learningcited by 6
  6. 2025Bridging Text and Crystal Structures: Literature-driven Contrastive Learning for Materials Sciencecited by 3
  7. 2025CAST: Cross Attention based multimodal fusion of Structure and Text for materials property predictioncited by 2
  8. 2025Learning simple heuristic rules for classifying materials based on chemical compositioncited by 2
  9. 2025Perspective on Utilizing Foundation Models for Laboratory Automation in Materials Researchcited by 2
  10. 2025Sparse, self-organizing ensembles of local kernels detect rare statistical anomaliescited by 1
  11. 2025MOCLIP: A Foundation Model for Large-Scale Nanophotonic Inverse Designcited by 1
  12. 2026MEIDNet: Multimodal generative AI framework for inverse materials designcited by 1
7 more
  1. 2025XxaCT-NN: Structure Agnostic Multimodal Learning for Materials Sciencecited by 0
  2. 2026Sustainable Materials Discovery in the Era of Artificial Intelligencecited by 0
  3. 2026SCALAR: Quantifying Structural Hallucination, Consistency, and Reasoning Gaps in Materials Foundation Modelscited by 0
  4. 2026Learning ORDER-Aware Multimodal Representations for Composite Materials Designcited by 0
  5. 2026Towards Automated Discovery: A Review of Generative Models, Multimodal Learning and Closed-Loop Workflows in Inverse Materials Designcited by 0
  6. 2026MatBind: A Shared Embedding Space for Multimodal Materials Characterizationcited by 0
  7. 2026PhyMo: A Physical-Field Modality for Multimodal AI4Physicscited by 0