# Training a Foundation Model for Materials on a Budget

arXiv: https://arxiv.org/abs/2508.16067 · HTML: https://mlsanity.com/paper/2508.16067/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 22 Aug 2025.

## Builds on (most cited first, 9 of 9)

- [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021)
- [EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations](https://mlsanity.com/paper/2306.12059/) (2023)
- [Equinox: neural networks in JAX via callable PyTrees and filtered transformations](https://mlsanity.com/paper/2111.00254/) (2021)
- [Learning Smooth and Expressive Interatomic Potentials for Physical Property Prediction](https://mlsanity.com/paper/2502.12147/) (2025)
- [Orb: A Fast, Scalable Neural Network Potential](https://mlsanity.com/paper/2410.22570/) (2024)
- [An Efficient Sparse Kernel Generator for O(3)-Equivariant Deep Networks](https://mlsanity.com/paper/2501.13986/) (2025)
- [High-performance training and inference for deep equivariant interatomic potentials](https://mlsanity.com/paper/2504.16068/) (2025)
- [The Price of Freedom: Exploring Expressivity and Runtime Tradeoffs in Equivariant Tensor Products](https://mlsanity.com/paper/2506.13523/) (2025)
- [A Materials Foundation Model via Hybrid Invariant-Equivariant Architectures](https://mlsanity.com/paper/2503.05771/) (2025)

## Built on by (most cited first, 6 of 6)

- [EquiformerV3: Scaling Efficient, Expressive, and General SE(3)-Equivariant Graph Attention Transformers](https://mlsanity.com/paper/2604.09130/) (2026)
- [MatRIS: Toward Reliable and Efficient Pretrained Machine Learning Interatomic Potentials](https://mlsanity.com/paper/2603.02002/) (2026)
- [PFT: Phonon Fine-tuning for Machine Learned Interatomic Potentials](https://mlsanity.com/paper/2601.07742/) (2026)
- [Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials](https://mlsanity.com/paper/2607.02499/) (2026)
- [A strategic roadmap for an atomistic machine-learning ecosystem](https://mlsanity.com/paper/2609.39090/) (2026)
- [Physics-Aligned Electronic Ground-State Learning Improves Generalization](https://mlsanity.com/paper/2610.10298/) (2026)

## Abstract (arXiv)

Foundation models for materials modeling are advancing quickly, but their training remains expensive, often placing state-of-the-art methods out of reach for many research groups. We introduce Nequix, a compact E(3)-equivariant potential that pairs a simplified NequIP design with modern training practices, including equivariant root-mean-square layer normalization and the Muon optimizer, to retain accuracy while substantially reducing compute requirements. Nequix has 700K parameters and was trained in 100 A100 GPU-hours. On the Matbench-Discovery and MDR Phonon benchmarks, Nequix ranks third overall while requiring a 20 times lower training cost than most other methods, and it delivers two orders of magnitude faster inference speed than the current top-ranked model. We release model weights and fully reproducible codebase at https://github.com/atomicarchitects/nequix.