mlsanityData to 7 Oct 2026

Scaling Laws and Symmetry, Evidence from Neural Force Fields

arXiv 2510.09768 ↗First posted 10 Oct 2025cs.LG cs.AI physics.comp-phcited by 4

Not on the map. Why?

Abstract (arXiv)

We present an empirical study in the geometric task of learning interatomic potentials, which shows equivariance matters even more at larger scales; we show a clear power-law scaling behaviour with respect to data, parameters and compute with ``architecture-dependent exponents''. In particular, we observe that equivariant architectures, which leverage task symmetry, scale better than non-equivariant models. Moreover, among equivariant architectures, higher-order representations translate to better scaling exponents. Our analysis also suggests that for compute-optimal training, the data and model sizes should scale in tandem regardless of the architecture. At a high level, these results suggest that, contrary to common belief, we should not leave it to the model to discover fundamental inductive biases such as symmetry, especially as we scale, because they change the inherent difficulty of the task and its scaling laws.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 54 of its 75 references that are in the map. First posted on arXiv 10 Oct 2025.

Builds on

54 of 75 references.

References · 54

  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  3. 2021Scaling Vision Transformerscited by 993
  4. 2021E(n) Equivariant Graph Neural Networkscited by 951
  5. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  6. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 701
  7. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  8. 2017Deep Learning Scaling is Predictable, Empiricallycited by 556
  9. 2020Directional Message Passing for Molecular Graphscited by 556
  10. 2020Scaling Laws for Autoregressive Generative Modelingcited by 510
  11. 2021Understanding over-squashing and bottlenecks on graphs via curvaturecited by 471
  12. 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 467
38 more
  1. 2023Scaling Data-Constrained Language Modelscited by 391
  2. 2021Explaining Neural Scaling Lawscited by 353
  3. 2021GemNet: Universal Directional Graph Neural Networks for Moleculescited by 342
  4. 2019Cormorant: Covariant Molecular Neural Networkscited by 262
  5. 2022Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphscited by 258
  6. 2020Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Moleculescited by 254
  7. 2023EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representationscited by 201
  8. 2019What graph neural networks cannot learn: depth vs widthcited by 195
  9. 2021Frame Averaging for Invariant and Equivariant Network Designcited by 166
  10. 2024The Road Less Scheduledcited by 158
  11. 2022Revisiting Neural Scaling Laws in Language and Visioncited by 129
  12. 2023Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNscited by 113
  13. 2022Equivariance with Learned Canonicalization Functionscited by 106
  14. 2024A Dynamical Model of Neural Scaling Lawscited by 105
  15. 2022Broken Neural Scaling Lawscited by 102
  16. 2022A Solvable Model of Neural Scaling Lawscited by 101
  17. 2025UMA: A Family of Universal Models for Atomscited by 99
  18. 2023On the Expressive Power of Geometric Graph Neural Networkscited by 89
  19. 2025Learning Smooth and Expressive Interatomic Potentials for Physical Property Predictioncited by 79
  20. 2023FAENet: Frame Averaging Equivariant GNN for Materials Modelingcited by 72
  21. 20244+3 Phases of Compute-Optimal Neural Scaling Lawscited by 66
  22. 2021Equivariant Contrastive Learningcited by 65
  23. 2020OrbNet: Deep Learning for Quantum Chemistry Using Symmetry-Adapted Atomic-Orbital Featurescited by 63
  24. 2023Swallowing the Bitter Pill: Simplified Scalable Conformer Generationcited by 55
  25. 2024Does equivariance matter at scale?cited by 49
  26. 2023Approximation-Generalization Trade-offs under (Approximate) Group Equivariancecited by 45
  27. 2024The Importance of Being Scalable: Improving the Speed and Accuracy of Neural Network Interatomic Potentials Across Chemical Domainscited by 40
  28. 2023Scaling MLPs: A Tale of Inductive Biascited by 37
  29. 2024On the Scalability of GNNs for Molecular Graphscited by 36
  30. 2023Learning Probabilistic Symmetrization for Architecture Agnostic Equivariancecited by 25
  31. 2023Lie Point Symmetry and Physics Informed Networkscited by 24
  32. 2024A Hitchhiker's Guide to Scaling Law Estimationcited by 23
  33. 2024The dark side of the forces: assessing non-conservative force models for atomistic machine learningcited by 23
  34. 2024Generalizing Denoising to Non-Equilibrium Structures Improves Equivariant Force Fieldscited by 18
  35. 2023Regularizing Towards Soft Equivariance Under Mixed Symmetriescited by 16
  36. 2024Symmetry-Informed Governing Equation Discoverycited by 14
  37. 2024Relaxed Equivariance via Multitask Learningcited by 13
  38. 2023Using Multiple Vector Channels Improves E(n)-Equivariant Graph Neural Networkscited by 10

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Same Architecture, Different Capacity: Optimizer-Induced Spectral Scaling Lawscited by 2
  2. 2026Coupled Cluster con M=oLe: Molecular Orbital Learning for Neural Wavefunctionscited by 0
  3. 2026Coupled Scaling: A Representational Accessibility Framework for Neural Scaling Lawscited by 0
  4. 2026Sample Complexity of Equivariant Reinforcement Learningcited by 0