mlsanityData to 14 Oct 2026

So3krates: Equivariant attention for interactions on arbitrary length-scales in molecular systems

arXiv 2205.14276 ↗First posted 28 May 2022cs.LGcited by 39

Not on the map. Why?

Abstract (arXiv)

The application of machine learning methods in quantum chemistry has enabled the study of numerous chemical phenomena, which are computationally intractable with traditional ab-initio methods. However, some quantum mechanical properties of molecules and materials depend on non-local electronic effects, which are often neglected due to the difficulty of modeling them efficiently. This work proposes a modified attention mechanism adapted to the underlying physics, which allows to recover the relevant non-local effects. Namely, we introduce spherical harmonic coordinates (SPHCs) to reflect higher-order geometric information for each atom in a molecule, enabling a non-local formulation of attention in the SPHC space. Our proposed model So3krates - a self-attention based message passing neural network - uncouples geometric information from atomic features, making them independently amenable to attention mechanisms. Thereby we construct spherical filters, which extend the concept of continuous filters in Euclidean space to SPHC space and serve as foundation for a spherical self-attention mechanism. We show that in contrast to other published methods, So3krates is able to describe non-local quantum mechanical effects over arbitrary length scales. Further, we find evidence that the inclusion of higher-order geometric correlations increases data efficiency and improves generalization. So3krates matches or exceeds state-of-the-art performance on popular benchmarks, notably, requiring a significantly lower number of parameters (0.25 - 0.4x) while at the same time giving a substantial speedup (6 - 14x for training and 2 - 11x for inference) compared to other models.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Builds on 26 of its 64 references that are in the map. First posted on arXiv 28 May 2022.

Builds on

26 of 64 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2017Graph Attention Networkscited by 8,923
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  6. 2017Non-local Neural Networkscited by 2,293
  7. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  8. 2016Group Equivariant Convolutional Networkscited by 1,165
  9. 2021E(n) Equivariant Graph Neural Networkscited by 965
  10. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 746
  11. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 708
  12. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 607
14 more
  1. 2020Directional Message Passing for Molecular Graphscited by 561
  2. 2021Equivariant message passing for the prediction of tensorial properties and molecular spectracited by 441
  3. 2021GemNet: Universal Directional Graph Neural Networks for Moleculescited by 343
  4. 2019Cormorant: Covariant Molecular Neural Networkscited by 263
  5. 2020Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densitiescited by 250
  6. 2020Machine Learning Force Fieldscited by 207
  7. 2022TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentialscited by 182
  8. 2018Point Convolutional Neural Networks by Extension Operatorscited by 154
  9. 2019Machine learning for molecular simulationcited by 110
  10. 2021SpookyNet: Learning Force Fields with Electronic Degrees of Freedom and Nonlocal Effectscited by 104
  11. 2022XAI for Transformers: Better Explanations through Conservative Propagationcited by 100
  12. 2021SE(3)-equivariant prediction of molecular wavefunctions and electronic densitiescited by 67
  13. 2017Hierarchical modeling of molecular energies using a deep neural networkcited by 37
  14. 2017Machine Learning Molecular Dynamics for the Simulation of Infrared Spectracited by 30

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 9; 2024: 14; 2025: 8; 2026: 7.

39 papers.

Sort:

By area:

  1. 2023FAENet: Frame Averaging Equivariant GNN for Materials Modelingcited by 73
  2. 2023A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systemscited by 68
  3. 2023Geometric Algebra Transformercited by 48
  4. 2023Ewald-based Long-Range Message Passing for Molecular Graphscited by 46
  5. 2024ET-Flow: Equivariant Flow-Matching for Molecular Conformer Generationcited by 43
  6. 2024The Importance of Being Scalable: Improving the Speed and Accuracy of Neural Network Interatomic Potentials Across Chemical Domainscited by 40
  7. 2024Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Productscited by 36
  8. 2023Navigating protein landscapes with a machine-learned transferable coarse-grained modelcited by 23
  9. 2024Are High-Degree Representations Really Unnecessary in Equivariant Graph Neural Networks?cited by 18
  10. 2024Beyond Canonicalization: How Tensorial Messages Improve Equivariant Message Passingcited by 15
  11. 2025Enhancing the Scalability and Applicability of Kohn-Sham Hamiltonians for Molecular Systemscited by 14
  12. 2023Euclidean, Projective, Conformal: Choosing a Geometric Algebra for Equivariant Transformerscited by 13
27 more
  1. 2024Higher-Rank Irreducible Cartesian Tensors for Equivariant Message Passingcited by 13
  2. 2024Probing the effects of broken symmetries in machine learningcited by 13
  3. 2024Euclidean Fast Attention -- Machine Learning Global Atomic Representations at Linear Costcited by 12
  4. 2026Pushing the limits of unconstrained machine-learned interatomic potentialscited by 11
  5. 2024Molecular relaxation by reverse diffusion with time step predictioncited by 9
  6. 2025Accurate Ab-initio Neural-network Solutions to Large-Scale Electronic Structure Problemscited by 8
  7. 2024Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Featurescited by 7
  8. 2026Learning Hamiltonian Flow Maps: Mean Flow Consistency for Large-Timestep Molecular Dynamicscited by 7
  9. 2024GeoMFormer: A General Architecture for Geometric Molecular Representation Learningcited by 6
  10. 2024Learning Equivariant Non-Local Electron Density Functionalscited by 6
  11. 2025Learning Non-Local Molecular Interactions via Equivariant Local Representations and Charge Equilibrationcited by 5
  12. 2023Equivariant Matrix Function Neural Networkscited by 3
  13. 2022Reconstructing Kernel-based Machine Learning Force Fields with Super-linear Convergencecited by 2
  14. 2025Representing spherical tensors with scalar-based machine-learning modelscited by 2
  15. 2025PRISM: Periodic Representation with multIscale and Similarity graph Modelling for enhanced crystal structure property predictioncited by 2
  16. 2023Stress and heat flux via automatic differentiationcited by 1
  17. 2023From Peptides to Nanostructures: A Euclidean Transformer for Fast and Stable Machine Learned Force Fieldscited by 1
  18. 2024FreeCG: Free the Design Space of Clebsch-Gordan Transform for Machine Learning Force Fieldscited by 1
  19. 2025Efficient Parallelization of Message Passing Neural Network Potentials for Large-scale Molecular Dynamicscited by 1
  20. 2025ADAPT: Lightweight, Long-Range Machine Learning Force Fields Without Graphscited by 1
  21. 2026Enhancing molecular dynamics with equivariant machine-learned densitiescited by 1
  22. 2026Generative Pseudo-Force Fields for Molecular Generationcited by 1
  23. 2026Machine Learning Multiscale Interactionscited by 1
  24. 2024Neural Polarization: Toward Electron Density for Molecules by Extending Equivariant Networkscited by 0
  25. 2025Machine learning surrogate models of many-body dispersion interactions in polymer meltscited by 0
  26. 2026Quantized SO(3)-Equivariant Graph Neural Networks for Efficient Molecular Property Predictioncited by 0
  27. 2026Preserving Continuous Symmetry in Discrete Spaces: Geometric-Aware Quantization for SO(3)-Equivariant GNNscited by 0