# Training Dynamics of Learning 3D-Rotational Equivariance

arXiv: https://arxiv.org/abs/2512.02303 · HTML: https://mlsanity.com/paper/2512.02303/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 30 references that are in the map.
- First posted on arXiv 2 Dec 2025.

## Builds on (most cited first, 25 of 25)

- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [Proteina: Scaling Flow-based Protein Structure Generative Models](https://mlsanity.com/paper/2503.00710/) (2025)
- [GemNet-OC: Developing Graph Neural Networks for Large and Diverse Molecular Simulation Datasets](https://mlsanity.com/paper/2204.02782/) (2022)
- [Swallowing the Bitter Pill: Simplified Scalable Conformer Generation](https://mlsanity.com/paper/2311.17932/) (2023)
- [The Lie Derivative for Measuring Learned Equivariance](https://mlsanity.com/paper/2210.02984/) (2022)
- [Does equivariance matter at scale?](https://mlsanity.com/paper/2410.23179/) (2024)
- [Neural Empirical Bayes](https://mlsanity.com/paper/1903.02334/) (2019)
- [Equivariant Adaptation of Large Pretrained Models](https://mlsanity.com/paper/2310.01647/) (2023)
- [3D molecule generation by denoising voxel grids](https://mlsanity.com/paper/2306.07473/) (2023)
- [The Importance of Being Scalable: Improving the Speed and Accuracy of Neural Network Interatomic Potentials Across Chemical Domains](https://mlsanity.com/paper/2410.24169/) (2024)
- [Smooth, exact rotational symmetrization for deep learning on point clouds](https://mlsanity.com/paper/2305.19302/) (2023)
- [From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction](https://mlsanity.com/paper/2310.16802/) (2023)
- [Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Products](https://mlsanity.com/paper/2401.10216/) (2024)
- [Convergence of stochastic gradient descent schemes for Lojasiewicz-landscapes](https://mlsanity.com/paper/2102.09385/) (2021)
- [Equivariance versus Augmentation for Spherical Images](https://mlsanity.com/paper/2202.03990/) (2022)
- [Improving Equivariant Model Training via Constraint Relaxation](https://mlsanity.com/paper/2408.13242/) (2024)
- [Optimization Dynamics of Equivariant and Augmented Neural Networks](https://mlsanity.com/paper/2303.13458/) (2023)
- [Relaxed Equivariance via Multitask Learning](https://mlsanity.com/paper/2410.17878/) (2024)
- [TABASCO: A Fast, Simplified Model for Molecular Generation with Improved Physical Quality](https://mlsanity.com/paper/2507.00899/) (2025)
- [Training or Architecture? How to Incorporate Invariance in Neural Networks](https://mlsanity.com/paper/2106.10044/) (2021)
- [In What Ways Are Deep Neural Networks Invariant and How Should We Measure This?](https://mlsanity.com/paper/2210.03773/) (2022)
- [Nutmeg and SPICE: Models and Data for Biomolecular Machine Learning](https://mlsanity.com/paper/2406.13112/) (2024)
- [NEBULA: Neural Empirical Bayes Under Latent Representations for Efficient and Controllable Design of Molecular Libraries](https://mlsanity.com/paper/2407.03428/) (2024)
- [Do we need equivariant models for molecule generation?](https://mlsanity.com/paper/2507.09753/) (2025)

## Built on by (most cited first, 1 of 1)

- [How unconstrained machine-learning models learn physical symmetries](https://mlsanity.com/paper/2603.24638/) (2026)

## Abstract (arXiv)

While data augmentation is widely used to train symmetry-agnostic models, it remains unclear how quickly and effectively they learn to respect symmetries. We investigate this by deriving a principled measure of equivariance error that, for convex losses, calculates the percent of total loss attributable to imperfections in learned symmetry. We focus our empirical investigation to 3D-rotation equivariance on high-dimensional molecular tasks (flow matching, force field prediction, denoising voxels) and find that models reduce equivariance error quickly to ≤2% held-out loss within 1k-10k training steps, a result robust to model and dataset size. This happens because learning 3D-rotational equivariance is an easier learning task, with a smoother and better-conditioned loss landscape, than the main prediction task. For 3D rotations, the loss penalty for non-equivariant models is small throughout training, so they may achieve lower test loss than equivariant models per GPU-hour unless the equivariant ``efficiency gap'' is narrowed. We also experimentally and theoretically investigate the relationships between relative equivariance error, learning gradients, and model parameters.