mlsanityData to 14 Oct 2026

Data efficiency and extrapolation trends in neural network interatomic potentials

arXiv 2302.05823 ↗First posted 12 Feb 2023cs.LG physics.chem-phcited by 10

Not on the map. Why?

Abstract (arXiv)

Over the last few years, key architectural advances have been proposed for neural network interatomic potentials (NNIPs), such as incorporating message-passing networks, equivariance, or many-body expansion terms. Although modern NNIP models exhibit small differences in energy/forces errors, improvements in accuracy are still considered the main target when developing new NNIP architectures. In this work, we show how architectural and optimization choices influence the generalization of NNIPs, revealing trends in molecular dynamics (MD) stability, data efficiency, and loss landscapes. Using the 3BPA dataset, we show that test errors in NNIP follow a scaling relation and can be robust to noise, but cannot predict MD stability in the high-accuracy regime. To circumvent this problem, we propose the use of loss landscape visualizations and a metric of loss entropy for predicting the generalization power of NNIPs. With a large-scale study on NequIP and MACE, we show that the loss entropy predicts out-of-distribution error and MD stability despite being computed only on the training set. Using this probe, we demonstrate how the choice of optimizers, loss function weighting, data normalization, and other architectural decisions influence the extrapolation behavior of NNIPs. Finally, we relate loss entropy to data efficiency, demonstrating that flatter landscapes also predict learning curve slopes. Our work provides a deep learning justification for the extrapolation performance of many common NNIPs, and introduces tools beyond accuracy metrics that can be used to inform the development of next-generation models.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 42 of its 94 references that are in the map. First posted on arXiv 12 Feb 2023.

Builds on

42 of 94 references.

References · 42

  1. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  2. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  3. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  4. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  5. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  6. 2019On the Convergence of Adam and Beyondcited by 841
  7. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 746
  8. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 708
  9. 2017Exploring Generalization in Deep Learningcited by 600
  10. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 582
  11. 2020Directional Message Passing for Molecular Graphscited by 561
  12. 2019Fantastic Generalization Measures and Where to Find Themcited by 513
30 more
  1. 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 474
  2. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 463
  3. 2021Equivariant message passing for the prediction of tensorial properties and molecular spectracited by 441
  4. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 359
  5. 2018How Does Batch Normalization Help Optimization?cited by 357
  6. 2021GemNet: Universal Directional Graph Neural Networks for Moleculescited by 343
  7. 2018Essentially No Barriers in Neural Network Energy Landscapecited by 330
  8. 2019Cormorant: Covariant Molecular Neural Networkscited by 263
  9. 2022Learning Local Equivariant Representations for Large-Scale Atomistic Dynamicscited by 256
  10. 2020Loss landscapes and optimization in over-parameterized non-linear systems and neural networkscited by 233
  11. 2017Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanicscited by 214
  12. 2020Machine Learning Force Fieldscited by 207
  13. 2014Qualitatively characterizing neural network optimization problemscited by 188
  14. 2018Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Networkcited by 171
  15. 2021A Universal Law of Robustness via Isoperimetrycited by 151
  16. 2021Exploring the Limits of Large Scale Pre-trainingcited by 104
  17. 2022The Design Space of E(3)-Equivariant Atom-Centered Interatomic Potentialscited by 101
  18. 2022Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulationscited by 95
  19. 2017The loss surface of deep and wide neural networkscited by 74
  20. 2019Visualizing and Understanding the Effectiveness of BERTcited by 67
  21. 2021ForceNet: A Graph Neural Network for Large-Scale Quantum Calculationscited by 58
  22. 2019Shaping the learning landscape in neural networks around wide flat minimacited by 47
  23. 2021A Loss Curvature Perspective on Training Instability in Deep Learningcited by 40
  24. 2017Hierarchical modeling of molecular energies using a deep neural networkcited by 37
  25. 2022SchNetPack 2.0: A neural network toolbox for atomistic machine learningcited by 29
  26. 2021Differentiable sampling of molecular geometries with uncertainty-based adversarial attackscited by 25
  27. 2016An empirical analysis of the optimization of deep network loss surfacescited by 21
  28. 2019Fast Neural Network Approach for Direct Covariant Forces Prediction in Complex Multi-Element Extended Systemscited by 12
  29. 2017Exploring loss function topology with cyclical learning ratescited by 11
  30. 2020Automated discovery of a robust interatomic potential for aluminumcited by 11

Built on by

Cited by year

Citing papers per year: 2024: 5; 2025: 5.

10 papers.

  1. 2024TorchMD-Net 2.0: Fast Neural Network Potentials for Molecular Simulationscited by 15
  2. 2024Model-free quantification of completeness, uncertainties, and outliers in atomistic machine learning using information theorycited by 10
  3. 2024Stability-Aware Training of Machine Learning Force Fields with Differentiable Boltzmann Estimatorscited by 8
  4. 2024LTAU-FF: Loss Trajectory Analysis for Uncertainty in Atomistic Force Fieldscited by 4
  5. 2024Enhanced sampling of robust molecular datasets with uncertainty-based collective variablescited by 2
  6. 2025Just One Layer Norm Guarantees Stable Extrapolationcited by 2
  7. 2025Breaking scaling relations with inverse catalysts: a machine learning exploration of trends in CO2 hydrogenation energy barrierscited by 1
  8. 2025Improving the Stability of GNN Force Field Models by Reducing Feature Correlationcited by 0
  9. 2025Ensemble Knowledge Distillation for Machine Learning Interatomic Potentialscited by 0
  10. 2025Maximizing Efficiency of Dataset Compression for Machine Learning Potentials With Information Theorycited by 0