# A recipe for scalable attention-based MLIPs: unlocking long-range accuracy with all-to-all node attention

arXiv: https://arxiv.org/abs/2603.06567 · HTML: https://mlsanity.com/paper/2603.06567/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 39 references that are in the map.
- First posted on arXiv 6 Mar 2026.

## Builds on (most cited first, 24 of 24)

- [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021)
- [SchNet: A continuous-filter convolutional neural network for modeling quantum interactions](https://mlsanity.com/paper/1706.08566/) (2017)
- [Directional Message Passing for Molecular Graphs](https://mlsanity.com/paper/2003.03123/) (2020)
- [MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields](https://mlsanity.com/paper/2206.07697/) (2022)
- [GemNet: Universal Directional Graph Neural Networks for Molecules](https://mlsanity.com/paper/2106.08903/) (2021)
- [Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs](https://mlsanity.com/paper/2206.11990/) (2022)
- [The Open Catalyst 2020 (OC20) Dataset and Community Challenges](https://mlsanity.com/paper/2010.09990/) (2020)
- [Machine Learning Force Fields](https://mlsanity.com/paper/2010.07067/) (2020)
- [EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations](https://mlsanity.com/paper/2306.12059/) (2023)
- [Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs](https://mlsanity.com/paper/2302.03655/) (2023)
- [UMA: A Family of Universal Models for Atoms](https://mlsanity.com/paper/2506.23971/) (2025)
- [The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalysts](https://mlsanity.com/paper/2206.08917/) (2022)
- [Learning Smooth and Expressive Interatomic Potentials for Physical Property Prediction](https://mlsanity.com/paper/2502.12147/) (2025)
- [SPICE, A Dataset of Drug-like Molecules and Peptides for Training Machine Learning Potentials](https://mlsanity.com/paper/2209.10702/) (2022)
- [GemNet-OC: Developing Graph Neural Networks for Large and Diverse Molecular Simulation Datasets](https://mlsanity.com/paper/2204.02782/) (2022)
- [Spherical Channels for Modeling Atomic Interactions](https://mlsanity.com/paper/2206.14331/) (2022)
- [The Importance of Being Scalable: Improving the Speed and Accuracy of Neural Network Interatomic Potentials Across Chemical Domains](https://mlsanity.com/paper/2410.24169/) (2024)
- [Latent Ewald summation for machine learning of long-range interactions](https://mlsanity.com/paper/2408.15165/) (2024)
- [PET-MAD, a lightweight universal interatomic potential for advanced materials modeling](https://mlsanity.com/paper/2503.14118/) (2025)
- [A predictive machine learning force field framework for liquid electrolyte development](https://mlsanity.com/paper/2404.07181/) (2024)
- [Euclidean Fast Attention -- Machine Learning Global Atomic Representations at Linear Cost](https://mlsanity.com/paper/2412.08541/) (2024)
- [A universal augmentation framework for long-range electrostatics in machine learning interatomic potentials](https://mlsanity.com/paper/2507.14302/) (2025)
- [Transformers Discover Molecular Structure Without Graph Priors](https://mlsanity.com/paper/2510.02259/) (2025)
- [From Evaluation to Design: Using Potential Energy Surface Smoothness Metrics to Guide Machine Learning Interatomic Potential Architectures](https://mlsanity.com/paper/2602.04861/) (2026)

## Built on by (most cited first, 8 of 8)

- [Knowing when to trust machine-learned interatomic potentials](https://mlsanity.com/paper/2605.00640/) (2026)
- [Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials](https://mlsanity.com/paper/2607.02499/) (2026)
- [Compact SO(3) Equivariant Atomistic Foundation Models via Structural Pruning](https://mlsanity.com/paper/2605.08885/) (2026)
- [Atomistic Modeling of Chemical Disorder in Materials: Bridging Conventional Methods and AI-Assisted Approaches](https://mlsanity.com/paper/2605.19124/) (2026)
- [Distilling latent electrostatics from foundation machine learning interatomic potentials](https://mlsanity.com/paper/2606.15001/) (2026)
- [E3J: An Efficient and Open-Source Backend for Euclidean Equivariant Operations on GPU and TPU](https://mlsanity.com/paper/2609.35099/) (2026)
- [A strategic roadmap for an atomistic machine-learning ecosystem](https://mlsanity.com/paper/2609.39090/) (2026)
- [NEMORA: Neural Equivariant Multipole Operators for Long-Range Atomistic Learning](https://mlsanity.com/paper/2610.10776/) (2026)

## Abstract (arXiv)

Machine-learning interatomic potentials (MLIPs) have advanced rapidly, with many top models relying on strong physics-based inductive biases. However, as models scale to larger systems like biomolecules and electrolytes, they struggle to accurately capture long-range (LR) interactions, leading current approaches to rely on explicit physics-based terms or components. In this work, we propose AllScAIP, a straightforward, attention-based, and energy-conserving MLIP model that scales to O(100 million) training samples. It addresses the long-range challenge using an all-to-all node attention component that is data-driven. Extensive ablations reveal that in low-data/small-model regimes, inductive biases improve sample efficiency. However, as data and model size scale, these benefits diminish or even reverse, while all-to-all attention remains critical for capturing LR interactions. Our model achieves state-of-the-art energy/force accuracy on molecular systems, as well as a number of physics-based evaluations (OMol25), while being competitive on materials (OMat24) and catalysts (OC20). Furthermore, it enables stable, long-timescale MD simulations that accurately recover experimental observables, including density and heat of vaporization predictions.