# CrysMMNet: Multimodal Representation for Crystal Property Prediction

arXiv: https://arxiv.org/abs/2307.05390 · HTML: https://mlsanity.com/paper/2307.05390/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 42 references that are in the map.
- First posted on arXiv 9 Jun 2023.

## Builds on (most cited first, 14 of 14)

- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [Benchmarking Graph Neural Networks](https://mlsanity.com/paper/2003.00982/) (2020)
- [FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling](https://mlsanity.com/paper/1801.10247/) (2018)
- [SchNet: A continuous-filter convolutional neural network for modeling quantum interactions](https://mlsanity.com/paper/1706.08566/) (2017)
- [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019)
- [Crystal Diffusion Variational Autoencoder for Periodic Material Generation](https://mlsanity.com/paper/2110.06197/) (2021)
- [A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay](https://mlsanity.com/paper/1803.09820/) (2018)
- [Periodic Graph Transformers for Crystal Material Property Prediction](https://mlsanity.com/paper/2209.11807/) (2022)
- [CrysGNN : Distilling pre-trained knowledge to enhance property prediction for crystalline materials](https://mlsanity.com/paper/2301.05852/) (2023)
- [Efficient, Interpretable Graph Neural Network Representation for Angle-dependent Properties and its Application to Optical Spectroscopy](https://mlsanity.com/paper/2109.11576/) (2021)

## Built on by (most cited first, 18 of 18)

- [LLM4Mat-Bench: Benchmarking Large Language Models for Materials Property Prediction](https://mlsanity.com/paper/2411.00177/) (2024)
- [Periodic Materials Generation using Text-Guided Joint Diffusion Model](https://mlsanity.com/paper/2503.00522/) (2025)
- [LLM Meets Diffusion: A Hybrid Framework for Crystal Material Generation](https://mlsanity.com/paper/2510.23040/) (2025)
- [MatTools: Benchmarking Large Language Models for Materials Science Tools](https://mlsanity.com/paper/2505.10852/) (2025)
- [M2Hub: Unlocking the Potential of Machine Learning for Materials Discovery](https://mlsanity.com/paper/2307.05378/) (2023)
- [Artificial Intelligence and Generative Models for Materials Discovery -- A Review](https://mlsanity.com/paper/2508.03278/) (2025)
- [xChemAgents: Agentic AI for Explainable Quantum Chemistry](https://mlsanity.com/paper/2505.20574/) (2025)
- [Text Takes Over: A Study of Modality Bias in Multimodal Intent Detection](https://mlsanity.com/paper/2508.16122/) (2025)
- [UniMat: Unifying Materials Embeddings through Multi-modal Learning](https://mlsanity.com/paper/2411.08664/) (2024)
- [CAST: Cross Attention based multimodal fusion of Structure and Text for materials property prediction](https://mlsanity.com/paper/2502.06836/) (2025)
- [Contrastive Learning of English Language and Crystal Graphs for Multimodal Representation of Materials Knowledge](https://mlsanity.com/paper/2502.16451/) (2025)
- [MatMMFuse: Multi-Modal Fusion model for Material Property Prediction](https://mlsanity.com/paper/2505.04634/) (2025)
- [MatMind: A Structure-Activity Knowledge-Driven Generative Foundation Model for Materials Science](https://mlsanity.com/paper/2606.07712/) (2026)
- [PDDFormer: Pairwise Distance Distribution Graph Transformer for Crystal Material Property Prediction](https://mlsanity.com/paper/2408.12984/) (2024)
- [MoMa: A Modular Deep Learning Framework for Material Property Prediction](https://mlsanity.com/paper/2502.15483/) (2025)
- [Multimodal Transformer for Sample-Aware Prediction of Metal-Organic Framework Properties](https://mlsanity.com/paper/2604.19383/) (2026)
- [Towards Automated Discovery: A Review of Generative Models, Multimodal Learning and Closed-Loop Workflows in Inverse Materials Design](https://mlsanity.com/paper/2606.02507/) (2026)
- [Model Agnostic Graph Prompt Learning for Crystal Property Prediction](https://mlsanity.com/paper/2607.08996/) (2026)

## Abstract (arXiv)

Machine Learning models have emerged as a powerful tool for fast and accurate prediction of different crystalline properties. Exiting state-of-the-art models rely on a single modality of crystal data i.e. crystal graph structure, where they construct multi-graph by establishing edges between nearby atoms in 3D space and apply GNN to learn materials representation. Thereby, they encode local chemical semantics around the atoms successfully but fail to capture important global periodic structural information like space group number, crystal symmetry, rotational information, etc, which influence different crystal properties. In this work, we leverage textual descriptions of materials to model global structural information into graph structure and learn a more robust and enriched representation of crystalline materials. To this effect, we first curate a textual dataset for crystalline material databases containing descriptions of each material. Further, we propose CrysMMNet, a simple multi-modal framework, which fuses both structural and textual representation together to generate a joint multimodal representation of crystalline materials. We conduct extensive experiments on two benchmark datasets across ten different properties to show that CrysMMNet outperforms existing state-of-the-art baseline methods with a good margin. We also observe that fusing the textual representation with crystal graph structure provides consistent improvement for all the SOTA GNN models compared to their own vanilla versions. We have shared the textual dataset, that we have curated for both the benchmark material databases, with the community for future use.