# GemNet-OC: Developing Graph Neural Networks for Large and Diverse Molecular Simulation Datasets

arXiv: https://arxiv.org/abs/2204.02782 · HTML: https://mlsanity.com/paper/2204.02782/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 49 references that are in the map.
- First posted on arXiv 6 Apr 2022.

## Builds on (most cited first, 28 of 28)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Predict then Propagate: Graph Neural Networks meet Personalized PageRank](https://mlsanity.com/paper/1810.05997/) (2018)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021)
- [On the Bottleneck of Graph Neural Networks and its Practical Implications](https://mlsanity.com/paper/2006.05205/) (2020)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [SchNet: A continuous-filter convolutional neural network for modeling quantum interactions](https://mlsanity.com/paper/1706.08566/) (2017)
- [Do Better ImageNet Models Transfer Better?](https://mlsanity.com/paper/1805.08974/) (2018)
- [Directional Message Passing for Molecular Graphs](https://mlsanity.com/paper/2003.03123/) (2020)
- [Equivariant message passing for the prediction of tensorial properties and molecular spectra](https://mlsanity.com/paper/2102.03150/) (2021)
- [GemNet: Universal Directional Graph Neural Networks for Molecules](https://mlsanity.com/paper/2106.08903/) (2021)
- [Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer](https://mlsanity.com/paper/2203.03466/) (2022)
- [Cormorant: Covariant Molecular Neural Networks](https://mlsanity.com/paper/1906.04015/) (2019)
- [Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules](https://mlsanity.com/paper/2011.14115/) (2020)
- [The Open Catalyst 2020 (OC20) Dataset and Community Challenges](https://mlsanity.com/paper/2010.09990/) (2020)
- [SpookyNet: Learning Force Fields with Electronic Degrees of Freedom and Nonlocal Effects](https://mlsanity.com/paper/2105.00304/) (2021)
- [ForceNet: A Graph Neural Network for Large-Scale Quantum Calculations](https://mlsanity.com/paper/2103.01436/) (2021)
- [A Close Look at Deep Learning with Small Data](https://mlsanity.com/paper/2003.12843/) (2020)
- [Rotation Invariant Graph Neural Networks using Spin Convolutions](https://mlsanity.com/paper/2106.09575/) (2021)
- [Molecular Mechanics-Driven Graph Neural Network with Multiplex Graph for Molecular Structures](https://mlsanity.com/paper/2011.07457/) (2020)
- [Towards Training Billion Parameter Graph Neural Networks for Atomic Simulations](https://mlsanity.com/paper/2203.09697/) (2022)
- [Small Data, Big Decisions: Model Selection in the Small-Data Regime](https://mlsanity.com/paper/2009.12583/) (2020)

## Built on by (most cited first, 30 of 61)

- [EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations](https://mlsanity.com/paper/2306.12059/) (2023)
- [Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs](https://mlsanity.com/paper/2302.03655/) (2023)
- [UMA: A Family of Universal Models for Atoms](https://mlsanity.com/paper/2506.23971/) (2025)
- [The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalysts](https://mlsanity.com/paper/2206.08917/) (2022)
- [Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations](https://mlsanity.com/paper/2210.07237/) (2022)
- [FAENet: Frame Averaging Equivariant GNN for Materials Modeling](https://mlsanity.com/paper/2305.05577/) (2023)
- [A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems](https://mlsanity.com/paper/2312.07511/) (2023)
- [Open Materials 2024 (OMat24) Inorganic Materials Dataset and Models](https://mlsanity.com/paper/2410.12771/) (2024)
- [Spherical Channels for Modeling Atomic Interactions](https://mlsanity.com/paper/2206.14331/) (2022)
- [Ewald-based Long-Range Message Passing for Molecular Graphs](https://mlsanity.com/paper/2303.04791/) (2023)
- [The Importance of Being Scalable: Improving the Speed and Accuracy of Neural Network Interatomic Potentials Across Chemical Domains](https://mlsanity.com/paper/2410.24169/) (2024)
- [From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction](https://mlsanity.com/paper/2310.16802/) (2023)
- [Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Products](https://mlsanity.com/paper/2401.10216/) (2024)
- [AdsorbML: A Leap in Efficiency for Adsorption Energy Calculations using Generalizable Machine Learning Potentials](https://mlsanity.com/paper/2211.16486/) (2022)
- [MOFDiff: Coarse-grained Diffusion for Metal-Organic Framework Design](https://mlsanity.com/paper/2310.10732/) (2023)
- [Discovering Symmetry Breaking in Physical Systems with Relaxed Group Convolution](https://mlsanity.com/paper/2310.02299/) (2023)
- [The Open DAC 2023 Dataset and Challenges for Sorbent Discovery in Direct Air Capture](https://mlsanity.com/paper/2311.00341/) (2023)
- [Generalizing Denoising to Non-Equilibrium Structures Improves Equivariant Force Fields](https://mlsanity.com/paper/2403.09549/) (2024)
- [DPA-1: Pretraining of Attention-based Deep Potential Model for Molecular Simulation](https://mlsanity.com/paper/2208.08236/) (2022)
- [Accurate machine learning force fields via experimental and simulation data fusion](https://mlsanity.com/paper/2308.09142/) (2023)
- [E2Former: An Efficient and Equivariant Transformer with Linear-Scaling Tensor Products](https://mlsanity.com/paper/2501.19216/) (2025)
- [Accelerating Molecular Graph Neural Networks via Knowledge Distillation](https://mlsanity.com/paper/2306.14818/) (2023)
- [On the Completeness of Invariant Geometric Deep Learning Models](https://mlsanity.com/paper/2402.04836/) (2024)
- [Efficient and Scalable Density Functional Theory Hamiltonian Prediction through Adaptive Sparsity](https://mlsanity.com/paper/2502.01171/) (2025)
- [AdsorbDiff: Adsorbate Placement via Conditional Denoising Diffusion](https://mlsanity.com/paper/2405.03962/) (2024)
- [Adsorb-Agent: Autonomous Identification of Stable Adsorption Configurations via Large Language Model Agent](https://mlsanity.com/paper/2410.16658/) (2024)
- [A recipe for scalable attention-based MLIPs: unlocking long-range accuracy with all-to-all node attention](https://mlsanity.com/paper/2603.06567/) (2026)
- [The Open DAC 2025 Dataset for Sorbent Discovery in Direct Air Capture](https://mlsanity.com/paper/2508.03162/) (2025)
- [MatRIS: Toward Reliable and Efficient Pretrained Machine Learning Interatomic Potentials](https://mlsanity.com/paper/2603.02002/) (2026)
- [PhAST: Physics-Aware, Scalable, and Task-specific GNNs for Accelerated Catalyst Design](https://mlsanity.com/paper/2211.12020/) (2022)

## Abstract (arXiv)

Recent years have seen the advent of molecular simulation datasets that are orders of magnitude larger and more diverse. These new datasets differ substantially in four aspects of complexity: 1. Chemical diversity (number of different elements), 2. system size (number of atoms per sample), 3. dataset size (number of data samples), and 4. domain shift (similarity of the training and test set). Despite these large differences, benchmarks on small and narrow datasets remain the predominant method of demonstrating progress in graph neural networks (GNNs) for molecular simulation, likely due to cheaper training compute requirements. This raises the question -- does GNN progress on small and narrow datasets translate to these more complex datasets? This work investigates this question by first developing the GemNet-OC model based on the large Open Catalyst 2020 (OC20) dataset. GemNet-OC outperforms the previous state-of-the-art on OC20 by 16% while reducing training time by a factor of 10. We then compare the impact of 18 model components and hyperparameter choices on performance in multiple datasets. We find that the resulting model would be drastically different depending on the dataset used for making model choices. To isolate the source of this discrepancy we study six subsets of the OC20 dataset that individually test each of the above-mentioned four dataset aspects. We find that results on the OC-2M subset correlate well with the full OC20 dataset while being substantially cheaper to train on. Our findings challenge the common practice of developing GNNs solely on small datasets, but highlight ways of achieving fast development cycles and generalizable results via moderately-sized, representative datasets such as OC-2M and efficient models such as GemNet-OC. Our code and pretrained model weights are open-sourced.