# Deconstructing equivariant representations in molecular systems

arXiv: https://arxiv.org/abs/2410.08131 · HTML: https://mlsanity.com/paper/2410.08131/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 10 Oct 2024.

## Builds on (most cited first, 16 of 16)

- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields](https://mlsanity.com/paper/2206.07697/) (2022)
- [Equivariant message passing for the prediction of tensorial properties and molecular spectra](https://mlsanity.com/paper/2102.03150/) (2021)
- [Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs](https://mlsanity.com/paper/2206.11990/) (2022)
- [Geometric and Physical Quantities Improve E(3) Equivariant Message Passing](https://mlsanity.com/paper/2110.02905/) (2021)
- [EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations](https://mlsanity.com/paper/2306.12059/) (2023)
- [Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs](https://mlsanity.com/paper/2302.03655/) (2023)
- [A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems](https://mlsanity.com/paper/2312.07511/) (2023)
- [Finding Symmetry Breaking Order Parameters with Euclidean Neural Networks](https://mlsanity.com/paper/2007.02005/) (2020)
- [Generalizing Denoising to Non-Equilibrium Structures Improves Equivariant Force Fields](https://mlsanity.com/paper/2403.09549/) (2024)
- [EGraFFBench: Evaluation of Equivariant Graph Neural Network Force Fields for Atomistic Simulations](https://mlsanity.com/paper/2310.02428/) (2023)
- [The Open MatSci ML Toolkit: A Flexible Framework for Machine Learning in Materials Science](https://mlsanity.com/paper/2210.17484/) (2022)

## Built on by (most cited first, 2 of 2)

- [ET-Flow: Equivariant Flow-Matching for Molecular Conformer Generation](https://mlsanity.com/paper/2410.22388/) (2024)
- [An Efficient Sparse Kernel Generator for O(3)-Equivariant Deep Networks](https://mlsanity.com/paper/2501.13986/) (2025)

## Abstract (arXiv)

Recent equivariant models have shown significant progress in not just chemical property prediction, but as surrogates for dynamical simulations of molecules and materials. Many of the top performing models in this category are built within the framework of tensor products, which preserves equivariance by restricting interactions and transformations to those that are allowed by symmetry selection rules. Despite being a core part of the modeling process, there has not yet been much attention into understanding what information persists in these equivariant representations, and their general behavior outside of benchmark metrics. In this work, we report on a set of experiments using a simple equivariant graph convolution model on the QM9 dataset, focusing on correlating quantitative performance with the resulting molecular graph embeddings. Our key finding is that, for a scalar prediction task, many of the irreducible representations are simply ignored during training -- specifically those pertaining to vector (l=1) and tensor quantities (l=2) -- an issue that does not necessarily make itself evident in the test metric. We empirically show that removing some unused orders of spherical harmonics improves model performance, correlating with improved latent space structure. We provide a number of recommendations for future experiments to try and improve efficiency and utilization of equivariant features based on these observations.