# How unconstrained machine-learning models learn physical symmetries

arXiv: https://arxiv.org/abs/2603.24638 · HTML: https://mlsanity.com/paper/2603.24638/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 76 references that are in the map.
- First posted on arXiv 25 Mar 2026.

## Builds on (most cited first, 30 of 33)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021)
- [Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets](https://mlsanity.com/paper/2201.02177/) (2022)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [Towards Understanding Grokking: An Effective Theory of Representation Learning](https://mlsanity.com/paper/2205.10343/) (2022)
- [Frame Averaging for Invariant and Equivariant Network Design](https://mlsanity.com/paper/2110.03336/) (2021)
- [Polar Transformer Networks](https://mlsanity.com/paper/1709.01889/) (2017)
- [FAENet: Frame Averaging Equivariant GNN for Materials Modeling](https://mlsanity.com/paper/2305.05577/) (2023)
- [Orb: A Fast, Scalable Neural Network Potential](https://mlsanity.com/paper/2410.22570/) (2024)
- [Geometric Deep Learning and Equivariant Neural Networks](https://mlsanity.com/paper/2105.13926/) (2021)
- [The Lie Derivative for Measuring Learned Equivariance](https://mlsanity.com/paper/2210.02984/) (2022)
- [Lattice gauge equivariant convolutional neural networks](https://mlsanity.com/paper/2012.12901/) (2020)
- [Smooth, exact rotational symmetrization for deep learning on point clouds](https://mlsanity.com/paper/2305.19302/) (2023)
- [PELICAN: Permutation Equivariant and Lorentz Invariant or Covariant Aggregator Network for Particle Physics](https://mlsanity.com/paper/2211.00454/) (2022)
- [The dark side of the forces: assessing non-conservative force models for atomistic machine learning](https://mlsanity.com/paper/2412.11569/) (2024)
- [PET-MAD, a lightweight universal interatomic potential for advanced materials modeling](https://mlsanity.com/paper/2503.14118/) (2025)
- [Equivariant representations for molecular Hamiltonians and N-center atomic-scale properties](https://mlsanity.com/paper/2109.12083/) (2021)
- [Incompleteness of graph neural networks for points clouds in three dimensions](https://mlsanity.com/paper/2201.07136/) (2022)
- [A General Framework for Equivariant Neural Networks on Reductive Lie Groups](https://mlsanity.com/paper/2306.00091/) (2023)
- [Probing the effects of broken symmetries in machine learning](https://mlsanity.com/paper/2406.17747/) (2024)
- [Relaxed Equivariance via Multitask Learning](https://mlsanity.com/paper/2410.17878/) (2024)
- [Pushing the limits of unconstrained machine-learned interatomic potentials](https://mlsanity.com/paper/2601.16195/) (2026)
- [How simple can you go? An off-the-shelf transformer approach to molecular dynamics](https://mlsanity.com/paper/2503.01431/) (2025)
- [Particle Trajectory Representation Learning with Masked Point Modeling](https://mlsanity.com/paper/2502.02558/) (2025)
- [Physics-Driven Learning for Inverse Problems in Quantum Chromodynamics](https://mlsanity.com/paper/2501.05580/) (2025)
- [Transformers Discover Molecular Structure Without Graph Priors](https://mlsanity.com/paper/2510.02259/) (2025)
- [PILArNet: Public Dataset for Particle Imaging Liquid Argon Detectors in High Energy Physics](https://mlsanity.com/paper/2006.01993/) (2020)
- [Fast evaluation of spherical harmonics with sphericart](https://mlsanity.com/paper/2302.08381/) (2023)

## Built on by (most cited first, 1 of 1)

- [Riemannian Flow Models with Reinforcement Learning for Molecular Crystal Structure Prediction](https://mlsanity.com/paper/2609.39773/) (2026)

## Abstract (arXiv)

The requirement of generating predictions that exactly fulfill the fundamental symmetry of the corresponding physical quantities has profoundly shaped the development of machine-learning models for physical simulations. In many cases, models are built using constrained mathematical forms that ensure that symmetries are enforced exactly. However, unconstrained models that do not obey rotational symmetries are often found to have competitive performance, and to be able to learn to a high level of accuracy an approximate equivariant behavior with a simple data augmentation strategy. In this paper, we introduce rigorous metrics to measure the symmetry content of the learned representations in such models, and assess the accuracy by which the outputs fulfill the equivariant condition. We apply these metrics to two unconstrained, transformer-based models operating on decorated point clouds (a graph neural network for atomistic simulations and a PointNet-style architecture for particle physics) to investigate how symmetry information is processed across architectural layers and is learned during training. Based on these insights, we establish a rigorous framework for diagnosing spectral failure modes in ML models. Enabled by this analysis, we demonstrate that one can achieve superior stability and accuracy by strategically injecting the minimum required inductive biases, preserving the high expressivity and scalability of unconstrained architectures while guaranteeing physical fidelity.