mlsanityData to 7 Oct 2026

E2Former: An Efficient and Equivariant Transformer with Linear-Scaling Tensor Products

arXiv 2501.19216 ↗First posted 31 Jan 2025cs.LG cond-mat.mtrl-scicited by 12

Not on the map. Why?

Abstract (arXiv)

Equivariant Graph Neural Networks (EGNNs) have demonstrated significant success in modeling microscale systems, including those in chemistry, biology and materials science. However, EGNNs face substantial computational challenges due to the high cost of constructing edge features via spherical tensor products, making them impractical for large-scale systems. To address this limitation, we introduce E2Former, an equivariant and efficient transformer architecture that incorporates the Wigner 6j convolution (Wigner 6j Conv). By shifting the computational burden from edges to nodes, the Wigner 6j Conv reduces the complexity from O(|E|) to O(| V|) while preserving both the model's expressive power and rotational equivariance. We show that this approach achieves a 7x-30x speedup compared to conventional SO(3) convolutions. Furthermore, our empirical results demonstrate that the derived E2Former mitigates the computational challenges of existing approaches without compromising the ability to capture detailed geometric information. This development could suggest a promising direction for scalable and efficient molecular modeling.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 30 of its 65 references that are in the map. First posted on arXiv 31 Jan 2025.

Builds on

30 of 65 references.

References · 30

  1. 2020Rethinking Attention with Performerscited by 1,584
  2. 2021E(n) Equivariant Graph Neural Networkscited by 951
  3. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  4. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 701
  5. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
  6. 2020Directional Message Passing for Molecular Graphscited by 556
  7. 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 467
  8. 2021Equivariant message passing for the prediction of tensorial properties and molecular spectracited by 438
  9. 2021GemNet: Universal Directional Graph Neural Networks for Moleculescited by 342
  10. 2020Learning from Protein Structure with Geometric Vector Perceptronscited by 308
  11. 2019Cormorant: Covariant Molecular Neural Networkscited by 262
  12. 2022Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphscited by 258
18 more
  1. 2022Learning Local Equivariant Representations for Large-Scale Atomistic Dynamicscited by 254
  2. 2020The Open Catalyst 2020 (OC20) Dataset and Community Challengescited by 246
  3. 2023EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representationscited by 201
  4. 2021Spherical Message Passing for 3D Graph Networkscited by 166
  5. 2023Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNscited by 113
  6. 2022The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalystscited by 95
  7. 2020On the Universality of Rotation Equivariant Point Cloud Networkscited by 80
  8. 2022SPICE, A Dataset of Drug-like Molecules and Peptides for Training Machine Learning Potentialscited by 80
  9. 2021SE(3) Equivariant Graph Neural Networks with Complete Local Framescited by 75
  10. 2022ComENet: Towards Complete and Efficient Message Passing for 3D Molecular Graphscited by 69
  11. 2022GemNet-OC: Developing Graph Neural Networks for Large and Diverse Molecular Simulation Datasetscited by 61
  12. 2023TensorNet: Cartesian Tensor Representations for Efficient Learning of Molecular Potentialscited by 53
  13. 2024The Importance of Being Scalable: Improving the Speed and Accuracy of Neural Network Interatomic Potentials Across Chemical Domainscited by 40
  14. 2024Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Productscited by 35
  15. 2020Finding Symmetry Breaking Order Parameters with Euclidean Neural Networkscited by 30
  16. 2018MT-CGCNN: Integrating Crystal Graph Convolutional Neural Network with Multitask Learning for Material Property Predictioncited by 17
  17. 2024Euclidean Fast Attention -- Machine Learning Global Atomic Representations at Linear Costcited by 12
  18. 2023E(3) Equivariant Graph Neural Networks for Particle-Based Fluid Mechanicscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 9.

12 papers.

  1. 2025Enhancing the Scalability and Applicability of Kohn-Sham Hamiltonians for Molecular Systemscited by 13
  2. 2025Efficient and Scalable Density Functional Theory Hamiltonian Prediction through Adaptive Sparsitycited by 10
  3. 2025Universally Invariant Learning in Equivariant GNNscited by 3
  4. 2026E2Former-V2: On-the-Fly Equivariant Attention with Linear Activation Memorycited by 3
  5. 2026Elign: Equivariant Diffusion Model Alignment from Foundational Machine Learning Force Fieldscited by 3
  6. 2026Edge Cluster Expansion with Radial Rotary Attention for Interatomic Potentialscited by 2
  7. 2026Integral Formulas for Vector Signal Tensor Productscited by 1
  8. 2026UBio-MolFM: A Universal Molecular Foundation Model for Bio-Systemscited by 0
  9. 2026Cross-Chirality Generalization by Axial Vectors for Hetero-Chiral Protein-Peptide Interaction Designcited by 0
  10. 2026Offline Materials Optimization with CliqueFlowmercited by 0
  11. 2026Spin-Weighted Spherical Harmonics Enable Complete and Scalable E(3)-Equivariant Networkscited by 0
  12. 2026E3J: An Efficient and Open-Source Backend for Euclidean Equivariant Operations on GPU and TPUcited by 0