# The Price of Freedom: Exploring Expressivity and Runtime Tradeoffs in Equivariant Tensor Products

arXiv: https://arxiv.org/abs/2506.13523 · HTML: https://mlsanity.com/paper/2506.13523/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 16 Jun 2025.

## Builds on (most cited first, 21 of 21)

- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021)
- [MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields](https://mlsanity.com/paper/2206.07697/) (2022)
- [3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data](https://mlsanity.com/paper/1807.02547/) (2018)
- [Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs](https://mlsanity.com/paper/2206.11990/) (2022)
- [Learning Local Equivariant Representations for Large-Scale Atomistic Dynamics](https://mlsanity.com/paper/2204.05249/) (2022)
- [FourCastNet: Accelerating Global High-Resolution Weather Forecasting using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2208.05419/) (2022)
- [e3nn: Euclidean Neural Networks](https://mlsanity.com/paper/2207.09453/) (2022)
- [Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network](https://mlsanity.com/paper/1806.09231/) (2018)
- [Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs](https://mlsanity.com/paper/2302.03655/) (2023)
- [N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials](https://mlsanity.com/paper/1803.01588/) (2018)
- [Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Products](https://mlsanity.com/paper/2401.10216/) (2024)
- [Efficient Generalized Spherical CNNs](https://mlsanity.com/paper/2010.11661/) (2020)
- [An Efficient Sparse Kernel Generator for O(3)-Equivariant Deep Networks](https://mlsanity.com/paper/2501.13986/) (2025)
- [Are High-Degree Representations Really Unnecessary in Equivariant Graph Neural Networks?](https://mlsanity.com/paper/2410.11443/) (2024)
- [Higher-Rank Irreducible Cartesian Tensors for Equivariant Message Passing](https://mlsanity.com/paper/2405.14253/) (2024)
- [Hierarchical Roofline Performance Analysis for Deep Learning Applications](https://mlsanity.com/paper/2009.05257/) (2020)
- [High-Rank Irreducible Cartesian Tensor Decomposition and Bases of Equivariant Spaces](https://mlsanity.com/paper/2412.18263/) (2024)
- [Complete and Efficient Covariants for 3D Point Configurations with Application to Learning Molecular Quantum Properties](https://mlsanity.com/paper/2409.02730/) (2024)
- [Hierarchical Roofline Analysis: How to Collect Data using Performance Tools on Intel CPUs and NVIDIA GPUs](https://mlsanity.com/paper/2009.02449/) (2020)
- [Optimizing Data Distribution and Kernel Performance for Efficient Training of Chemistry Foundation Models: A Case Study with MACE](https://mlsanity.com/paper/2504.10700/) (2025)

## Built on by (most cited first, 15 of 15)

- [EquiformerV3: Scaling Efficient, Expressive, and General SE(3)-Equivariant Graph Attention Transformers](https://mlsanity.com/paper/2604.09130/) (2026)
- [Training a Foundation Model for Materials on a Budget](https://mlsanity.com/paper/2508.16067/) (2025)
- [Universally Invariant Learning in Equivariant GNNs](https://mlsanity.com/paper/2510.13169/) (2025)
- [A Cartesian-3j Framework for Machine Learning Interatomic Potentials](https://mlsanity.com/paper/2512.16882/) (2025)
- [E2Former-V2: On-the-Fly Equivariant Attention with Linear Activation Memory](https://mlsanity.com/paper/2601.16622/) (2026)
- [How the Optimizer Shapes Learned Solutions in Equivariant Neural Networks](https://mlsanity.com/paper/2605.27662/) (2026)
- [Tensor Decomposition Networks for Fast Machine Learning Interatomic Potential Computations](https://mlsanity.com/paper/2507.01131/) (2025)
- [Integral Formulas for Vector Signal Tensor Products](https://mlsanity.com/paper/2603.08630/) (2026)
- [Sobek: Streaming Equivariant Tensor Product Convolutions](https://mlsanity.com/paper/2607.18074/) (2026)
- [Diagnosing Spectral Ceilings in Equivariant Neural Force Fields](https://mlsanity.com/paper/2605.08286/) (2026)
- [MALOQ: Massively Accelerated Learning of Operators for Quantum Transport](https://mlsanity.com/paper/2606.28911/) (2026)
- [Spin-Weighted Spherical Harmonics Enable Complete and Scalable E(3)-Equivariant Networks](https://mlsanity.com/paper/2607.01408/) (2026)
- [E3DGS: Unified Geometric-Photometric Equivariance for 3D Gaussian Splatting via Color-as-Geometry Embedding](https://mlsanity.com/paper/2607.15536/) (2026)
- [E3J: An Efficient and Open-Source Backend for Euclidean Equivariant Operations on GPU and TPU](https://mlsanity.com/paper/2609.35099/) (2026)
- [LunaTP: Enabling Efficient and Scalable Tensor Product for Equivariant MLIPs](https://mlsanity.com/paper/2610.18393/) (2026)

## Abstract (arXiv)

E(3)-equivariant neural networks have demonstrated success across a wide range of 3D modelling tasks. A fundamental operation in these networks is the tensor product, which interacts two geometric features in an equivariant manner to create new features. Due to the high computational complexity of the tensor product, significant effort has been invested to optimize the runtime of this operation. For example, Luo et al. (2024) recently proposed the Gaunt tensor product (GTP) which promises a significant speedup. In this work, we provide a careful, systematic analysis of a number of tensor product operations. In particular, we emphasize that different tensor products are not performing the same operation. The reported speedups typically come at the cost of expressivity. We introduce measures of expressivity and interactability to characterize these differences. In addition, we realized the original implementation of GTP can be greatly simplified by directly using a spherical grid at no cost in asymptotic runtime. This spherical grid approach is faster on our benchmarks and in actual training of the MACE interatomic potential by 30%. Finally, we provide the first systematic microbenchmarks of the various tensor product operations. We find that the theoretical runtime guarantees can differ wildly from empirical performance, demonstrating the need for careful application-specific benchmarking. Code is available at https://github.com/atomicarchitects/PriceofFreedom.