# Complete and Efficient Graph Transformers for Crystal Material Property Prediction

arXiv: https://arxiv.org/abs/2403.11857 · HTML: https://mlsanity.com/paper/2403.11857/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 54 references that are in the map.
- First posted on arXiv 18 Mar 2024.

## Builds on (most cited first, 30 of 30)

- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021)
- [SchNet: A continuous-filter convolutional neural network for modeling quantum interactions](https://mlsanity.com/paper/1706.08566/) (2017)
- [Directional Message Passing for Molecular Graphs](https://mlsanity.com/paper/2003.03123/) (2020)
- [Graph U-Nets](https://mlsanity.com/paper/1905.05178/) (2019)
- [GemNet: Universal Directional Graph Neural Networks for Molecules](https://mlsanity.com/paper/2106.08903/) (2021)
- [Crystal Diffusion Variational Autoencoder for Periodic Material Generation](https://mlsanity.com/paper/2110.06197/) (2021)
- [Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs](https://mlsanity.com/paper/2206.11990/) (2022)
- [The Open Catalyst 2020 (OC20) Dataset and Community Challenges](https://mlsanity.com/paper/2010.09990/) (2020)
- [Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates](https://mlsanity.com/paper/1708.07120/) (2017)
- [GraphDF: A Discrete Flow Model for Molecular Graph Generation](https://mlsanity.com/paper/2102.01189/) (2021)
- [e3nn: Euclidean Neural Networks](https://mlsanity.com/paper/2207.09453/) (2022)
- [Spherical Message Passing for 3D Graph Networks](https://mlsanity.com/paper/2102.05013/) (2021)
- [Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems](https://mlsanity.com/paper/2307.08423/) (2023)
- [Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs](https://mlsanity.com/paper/2302.03655/) (2023)
- [Periodic Graph Transformers for Crystal Material Property Prediction](https://mlsanity.com/paper/2209.11807/) (2022)
- [GraphEBM: Molecular Graph Generation with Energy-Based Models](https://mlsanity.com/paper/2102.00546/) (2021)
- [DIG: A Turnkey Library for Diving into Graph Deep Learning Research](https://mlsanity.com/paper/2103.12608/) (2021)
- [ComENet: Towards Complete and Efficient Message Passing for 3D Molecular Graphs](https://mlsanity.com/paper/2206.08515/) (2022)
- [Learning Hierarchical Protein Representations via Complete 3D Graph Networks](https://mlsanity.com/paper/2207.12600/) (2022)
- [Efficient Approximations of Complete Interatomic Potentials for Crystal Property Prediction](https://mlsanity.com/paper/2306.10045/) (2023)
- [GraphFM: Improving Large-Scale GNN Training via Feature Momentum](https://mlsanity.com/paper/2206.07161/) (2022)
- [Informing Geometric Deep Learning with Electronic Interactions to Accelerate Quantum Chemistry](https://mlsanity.com/paper/2105.14655/) (2021)
- [A Latent Diffusion Model for Protein Structure Generation](https://mlsanity.com/paper/2305.04120/) (2023)
- [Connectivity Optimized Nested Graph Networks for Crystal Structures](https://mlsanity.com/paper/2302.14102/) (2023)
- [Machine Learning Enabled Computational Screening of Inorganic Solid Electrolytes for Dendrite Suppression with Li Metal Anode](https://mlsanity.com/paper/1804.04651/) (2018)
- [Fast Quantum Property Prediction via Deeper 2D and 3D Graph Networks](https://mlsanity.com/paper/2106.08551/) (2021)
- [Lattice Convolutional Networks for Learning Ground States of Quantum Many-Body Systems](https://mlsanity.com/paper/2206.07370/) (2022)
- [The Density Fingerprint of a Periodic Point Set](https://mlsanity.com/paper/2104.11046/) (2021)
- [Fast predictions of lattice energies by continuous isometry invariants of crystal structures](https://mlsanity.com/paper/2108.07233/) (2021)

## Built on by (most cited first, 30 of 34)

- [From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery](https://mlsanity.com/paper/2508.14111/) (2025)
- [Equivariance via Minimal Frame Averaging for More Symmetries and Efficiency](https://mlsanity.com/paper/2406.07598/) (2024)
- [A Survey of Graph Transformers: Architectures, Theories and Applications](https://mlsanity.com/paper/2502.16533/) (2025)
- [Invariant Tokenization of Crystalline Materials for Language Model Enabled Generation](https://mlsanity.com/paper/2503.00152/) (2025)
- [Toward Greater Autonomy in Materials Discovery Agents: Unifying Planning, Physics, and Scientists](https://mlsanity.com/paper/2506.05616/) (2025)
- [A Space Group Symmetry Informed Network for O(3) Equivariant Crystal Tensor Prediction](https://mlsanity.com/paper/2406.12888/) (2024)
- [Generative Inverse Design of Crystal Structures via Diffusion Models with Transformers](https://mlsanity.com/paper/2406.09263/) (2024)
- [A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Tools](https://mlsanity.com/paper/2506.20743/) (2025)
- [A Materials Foundation Model via Hybrid Invariant-Equivariant Architectures](https://mlsanity.com/paper/2503.05771/) (2025)
- [Beyond Structure: Invariant Crystal Property Prediction with Pseudo-Particle Ray Diffraction](https://mlsanity.com/paper/2509.21778/) (2025)
- [CrysToGraph: A Comprehensive Predictive Model for Crystal Materials Properties and the Benchmark](https://mlsanity.com/paper/2407.16131/) (2024)
- [CLOUD: A Scalable and Physics-Informed Foundation Model for Crystal Representation Learning](https://mlsanity.com/paper/2506.17345/) (2025)
- [Material Property Prediction with Element Attribute Knowledge Graphs and Multimodal Representation Learning](https://mlsanity.com/paper/2411.08414/) (2024)
- [UniMate: A Unified Model for Mechanical Metamaterial Generation, Property Prediction, and Condition Confirmation](https://mlsanity.com/paper/2506.15722/) (2025)
- [PRISM: Periodic Representation with multIscale and Similarity graph Modelling for enhanced crystal structure property prediction](https://mlsanity.com/paper/2511.20362/) (2025)
- [ReciNet: Reciprocal Space-Aware Long-Range Modeling for Crystalline Property Prediction](https://mlsanity.com/paper/2502.02748/) (2025)
- [Machine Learning - Driven Materials Discovery: Unlocking Next-Generation Functional Materials - A review](https://mlsanity.com/paper/2503.18975/) (2025)
- [FIRE-GNN: Force-informed, Relaxed Equivariance Graph Neural Network for Rapid and Accurate Prediction of Surface Properties](https://mlsanity.com/paper/2508.16012/) (2025)
- [Benchmarking GNNs for OOD Materials Property Prediction with Uncertainty Quantification](https://mlsanity.com/paper/2511.11697/) (2025)
- [A Single Architecture for Representing Invariance Under Any Space Group](https://mlsanity.com/paper/2512.13989/) (2025)
- [Efficient Equivariant High-Order Crystal Tensor Prediction via Cartesian Local-Environment Many-Body Coupling](https://mlsanity.com/paper/2602.04323/) (2026)
- [Machine Learning Multiscale Interactions](https://mlsanity.com/paper/2605.25710/) (2026)
- [Latent Diffusion Pretraining for Crystal Property Prediction](https://mlsanity.com/paper/2606.00776/) (2026)
- [Towards Efficient Large Language Models for Scientific Text: A Review](https://mlsanity.com/paper/2408.10729/) (2024)
- [PDDFormer: Pairwise Distance Distribution Graph Transformer for Crystal Material Property Prediction](https://mlsanity.com/paper/2408.12984/) (2024)
- [MoMa: A Modular Deep Learning Framework for Material Property Prediction](https://mlsanity.com/paper/2502.15483/) (2025)
- [CrystalFramer: Rethinking the Role of Frames for SE(3)-Invariant Crystal Structure Modeling](https://mlsanity.com/paper/2503.02209/) (2025)
- [Quotient Complex Transformer (QCformer) for Perovskite Data Analysis](https://mlsanity.com/paper/2505.09174/) (2025)
- [Universal crystal material property prediction via multi-view geometric fusion in graph transformers](https://mlsanity.com/paper/2507.15303/) (2025)
- [Challenges in Non-Polymeric Crystal Structure Prediction: Why a Geometric, Permutation-Invariant Loss is Needed](https://mlsanity.com/paper/2509.00832/) (2025)

## Abstract (arXiv)

Crystal structures are characterized by atomic bases within a primitive unit cell that repeats along a regular lattice throughout 3D space. The periodic and infinite nature of crystals poses unique challenges for geometric graph representation learning. Specifically, constructing graphs that effectively capture the complete geometric information of crystals and handle chiral crystals remains an unsolved and challenging problem. In this paper, we introduce a novel approach that utilizes the periodic patterns of unit cells to establish the lattice-based representation for each atom, enabling efficient and expressive graph representations of crystals. Furthermore, we propose ComFormer, a SE(3) transformer designed specifically for crystalline materials. ComFormer includes two variants; namely, iComFormer that employs invariant geometric descriptors of Euclidean distances and angles, and eComFormer that utilizes equivariant vector representations. Experimental results demonstrate the state-of-the-art predictive accuracy of ComFormer variants on various tasks across three widely-used crystal benchmarks. Our code is publicly available as part of the AIRS library (https://github.com/divelab/AIRS).