# E2Former: An Efficient and Equivariant Transformer with Linear-Scaling Tensor Products

arXiv: https://arxiv.org/abs/2501.19216 · HTML: https://mlsanity.com/paper/2501.19216/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 65 references that are in the map.
- First posted on arXiv 31 Jan 2025.

## Builds on (most cited first, 30 of 30)

- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [Directional Message Passing for Molecular Graphs](https://mlsanity.com/paper/2003.03123/) (2020)
- [MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields](https://mlsanity.com/paper/2206.07697/) (2022)
- [Equivariant message passing for the prediction of tensorial properties and molecular spectra](https://mlsanity.com/paper/2102.03150/) (2021)
- [GemNet: Universal Directional Graph Neural Networks for Molecules](https://mlsanity.com/paper/2106.08903/) (2021)
- [Learning from Protein Structure with Geometric Vector Perceptrons](https://mlsanity.com/paper/2009.01411/) (2020)
- [Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs](https://mlsanity.com/paper/2206.11990/) (2022)
- [Cormorant: Covariant Molecular Neural Networks](https://mlsanity.com/paper/1906.04015/) (2019)
- [Learning Local Equivariant Representations for Large-Scale Atomistic Dynamics](https://mlsanity.com/paper/2204.05249/) (2022)
- [The Open Catalyst 2020 (OC20) Dataset and Community Challenges](https://mlsanity.com/paper/2010.09990/) (2020)
- [EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations](https://mlsanity.com/paper/2306.12059/) (2023)
- [Spherical Message Passing for 3D Graph Networks](https://mlsanity.com/paper/2102.05013/) (2021)
- [Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs](https://mlsanity.com/paper/2302.03655/) (2023)
- [The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalysts](https://mlsanity.com/paper/2206.08917/) (2022)
- [SPICE, A Dataset of Drug-like Molecules and Peptides for Training Machine Learning Potentials](https://mlsanity.com/paper/2209.10702/) (2022)
- [On the Universality of Rotation Equivariant Point Cloud Networks](https://mlsanity.com/paper/2010.02449/) (2020)
- [SE(3) Equivariant Graph Neural Networks with Complete Local Frames](https://mlsanity.com/paper/2110.14811/) (2021)
- [ComENet: Towards Complete and Efficient Message Passing for 3D Molecular Graphs](https://mlsanity.com/paper/2206.08515/) (2022)
- [GemNet-OC: Developing Graph Neural Networks for Large and Diverse Molecular Simulation Datasets](https://mlsanity.com/paper/2204.02782/) (2022)
- [TensorNet: Cartesian Tensor Representations for Efficient Learning of Molecular Potentials](https://mlsanity.com/paper/2306.06482/) (2023)
- [The Importance of Being Scalable: Improving the Speed and Accuracy of Neural Network Interatomic Potentials Across Chemical Domains](https://mlsanity.com/paper/2410.24169/) (2024)
- [Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Products](https://mlsanity.com/paper/2401.10216/) (2024)
- [Finding Symmetry Breaking Order Parameters with Euclidean Neural Networks](https://mlsanity.com/paper/2007.02005/) (2020)
- [MT-CGCNN: Integrating Crystal Graph Convolutional Neural Network with Multitask Learning for Material Property Prediction](https://mlsanity.com/paper/1811.05660/) (2018)
- [Euclidean Fast Attention -- Machine Learning Global Atomic Representations at Linear Cost](https://mlsanity.com/paper/2412.08541/) (2024)
- [E(3) Equivariant Graph Neural Networks for Particle-Based Fluid Mechanics](https://mlsanity.com/paper/2304.00150/) (2023)

## Built on by (most cited first, 12 of 12)

- [Enhancing the Scalability and Applicability of Kohn-Sham Hamiltonians for Molecular Systems](https://mlsanity.com/paper/2502.19227/) (2025)
- [Efficient and Scalable Density Functional Theory Hamiltonian Prediction through Adaptive Sparsity](https://mlsanity.com/paper/2502.01171/) (2025)
- [Universally Invariant Learning in Equivariant GNNs](https://mlsanity.com/paper/2510.13169/) (2025)
- [E2Former-V2: On-the-Fly Equivariant Attention with Linear Activation Memory](https://mlsanity.com/paper/2601.16622/) (2026)
- [Elign: Equivariant Diffusion Model Alignment from Foundational Machine Learning Force Fields](https://mlsanity.com/paper/2601.21985/) (2026)
- [Edge Cluster Expansion with Radial Rotary Attention for Interatomic Potentials](https://mlsanity.com/paper/2607.10664/) (2026)
- [Integral Formulas for Vector Signal Tensor Products](https://mlsanity.com/paper/2603.08630/) (2026)
- [UBio-MolFM: A Universal Molecular Foundation Model for Bio-Systems](https://mlsanity.com/paper/2602.17709/) (2026)
- [Cross-Chirality Generalization by Axial Vectors for Hetero-Chiral Protein-Peptide Interaction Design](https://mlsanity.com/paper/2602.20176/) (2026)
- [Offline Materials Optimization with CliqueFlowmer](https://mlsanity.com/paper/2603.06082/) (2026)
- [Spin-Weighted Spherical Harmonics Enable Complete and Scalable E(3)-Equivariant Networks](https://mlsanity.com/paper/2607.01408/) (2026)
- [E3J: An Efficient and Open-Source Backend for Euclidean Equivariant Operations on GPU and TPU](https://mlsanity.com/paper/2609.35099/) (2026)

## Abstract (arXiv)

Equivariant Graph Neural Networks (EGNNs) have demonstrated significant success in modeling microscale systems, including those in chemistry, biology and materials science. However, EGNNs face substantial computational challenges due to the high cost of constructing edge features via spherical tensor products, making them impractical for large-scale systems. To address this limitation, we introduce E2Former, an equivariant and efficient transformer architecture that incorporates the Wigner 6j convolution (Wigner 6j Conv). By shifting the computational burden from edges to nodes, the Wigner 6j Conv reduces the complexity from O(|E|) to  O(| V|) while preserving both the model's expressive power and rotational equivariance. We show that this approach achieves a 7x-30x speedup compared to conventional SO(3) convolutions. Furthermore, our empirical results demonstrate that the derived E2Former mitigates the computational challenges of existing approaches without compromising the ability to capture detailed geometric information. This development could suggest a promising direction for scalable and efficient molecular modeling.