# Formula graph self-attention network for representation-domain independent materials discovery

arXiv: https://arxiv.org/abs/2201.05649 · HTML: https://mlsanity.com/paper/2201.05649/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 58 references that are in the map.
- First posted on arXiv 14 Jan 2022.

## Builds on (most cited first, 7 of 7)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [OrbNet: Deep Learning for Quantum Chemistry Using Symmetry-Adapted Atomic-Orbital Features](https://mlsanity.com/paper/2007.08026/) (2020)
- [Predicting materials properties without crystal structure: Deep representation learning from stoichiometry](https://mlsanity.com/paper/1910.00617/) (2019)
- [Scalable deeper graph neural networks for high-performance materials property prediction](https://mlsanity.com/paper/2109.12283/) (2021)
- [Distributed Representations of Atoms and Materials for Machine Learning](https://mlsanity.com/paper/2107.14664/) (2021)
- [Materials Representation and Transfer Learning for Multi-Property Prediction](https://mlsanity.com/paper/2106.02225/) (2021)
- [Analogical discovery of disordered perovskite oxides by crystal structure information hidden in unsupervised material fingerprints](https://mlsanity.com/paper/2105.11877/) (2021)

## Built on by (most cited first, 7 of 7)

- [Crystal Twins: Self-supervised Learning for Crystalline Material Property Prediction](https://mlsanity.com/paper/2205.01893/) (2022)
- [DenseGNN: universal and scalable deeper graph neural networks for high-performance property prediction in crystals and molecules](https://mlsanity.com/paper/2501.03278/) (2025)
- [Materials Informatics Transformer: A Language Model for Interpretable Materials Properties Prediction](https://mlsanity.com/paper/2308.16259/) (2023)
- [CLOUD: A Scalable and Physics-Informed Foundation Model for Crystal Representation Learning](https://mlsanity.com/paper/2506.17345/) (2025)
- [Transfer Learning for Deep Learning-based Prediction of Lattice Thermal Conductivity](https://mlsanity.com/paper/2411.18259/) (2024)
- [SA-GAT-SR: Self-Adaptable Graph Attention Networks with Symbolic Regression for high-fidelity material property prediction](https://mlsanity.com/paper/2505.00625/) (2025)
- [Optimizing Expert-Designed Crystal Graph Networks for Band-Gap Prediction with an Autonomous LLM Research Loop](https://mlsanity.com/paper/2606.29717/) (2026)

## Abstract (arXiv)

The success of machine learning (ML) in materials property prediction depends heavily on how the materials are represented for learning. Two dominant families of material descriptors exist, one that encodes crystal structure in the representation and the other that only uses stoichiometric information with the hope of discovering new materials. Graph neural networks (GNNs) in particular have excelled in predicting material properties within chemical accuracy. However, current GNNs are limited to only one of the above two avenues owing to the little overlap between respective material representations. Here, we introduce a new concept of formula graph which unifies stoichiometry-only and structure-based material descriptors. We further develop a self-attention integrated GNN that assimilates a formula graph and show that the proposed architecture produces material embeddings transferable between the two domains. Our model can outperform some previously proposed structure-agnostic models and their structure-based counterparts while exhibiting better sample efficiency and faster convergence. Finally, the model is applied in a challenging exemplar to predict the complex dielectric function of materials and nominate new substances that potentially exhibit epsilon-near-zero phenomena.