# Towards fully covariant machine learning arXiv: https://arxiv.org/abs/2301.13724 ยท HTML: https://mlsanity.com/paper/2301.13724/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Builds on 41 of its 93 references that are in the map. - First posted on arXiv 31 Jan 2023. ## Builds on (most cited first, 30 of 41) - [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017) - [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021) - [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016) - [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013) - [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018) - [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015) - [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018) - [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021) - [Deep Double Descent: Where Bigger Models and More Data Hurt](https://mlsanity.com/paper/1912.02292/) (2019) - [Benign Overfitting in Linear Regression](https://mlsanity.com/paper/1906.11300/) (2019) - [Sharp Minima Can Generalize For Deep Nets](https://mlsanity.com/paper/1703.04933/) (2017) - [Invariant and Equivariant Graph Networks](https://mlsanity.com/paper/1812.09902/) (2018) - [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018) - [Learning Local Equivariant Representations for Large-Scale Atomistic Dynamics](https://mlsanity.com/paper/2204.05249/) (2022) - [Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data](https://mlsanity.com/paper/2002.12880/) (2020) - [A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups](https://mlsanity.com/paper/2104.09459/) (2021) - [e3nn: Euclidean Neural Networks](https://mlsanity.com/paper/2207.09453/) (2022) - [EquiBind: Geometric Deep Learning for Drug Binding Structure Prediction](https://mlsanity.com/paper/2202.05146/) (2022) - [Scalars are universal: Equivariant machine learning, structured like classical physics](https://mlsanity.com/paper/2106.06610/) (2021) - [Equivariance with Learned Canonicalization Functions](https://mlsanity.com/paper/2211.06489/) (2022) - [Hamiltonian Graph Networks with ODE Integrators](https://mlsanity.com/paper/1909.12790/) (2019) - [ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA](https://mlsanity.com/paper/2002.11537/) (2020) - [Approximately Equivariant Networks for Imperfectly Symmetric Dynamics](https://mlsanity.com/paper/2201.11969/) (2022) - [Provably Strict Generalisation Benefit for Equivariant Models](https://mlsanity.com/paper/2102.10333/) (2021) - [Learning with invariances in random features and kernel models](https://mlsanity.com/paper/2102.13219/) (2021) - [Relative Flatness and Generalization](https://mlsanity.com/paper/2001.00939/) (2020) - [Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds](https://mlsanity.com/paper/2106.06020/) (2021) - [Function Classes for Identifiable Nonlinear Independent Component Analysis](https://mlsanity.com/paper/2208.06406/) (2022) - [N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials](https://mlsanity.com/paper/1803.01588/) (2018) - [Low Dimensional Invariant Embeddings for Universal Geometric Learning](https://mlsanity.com/paper/2205.02956/) (2022) ## Built on by (most cited first, 10 of 10) - [Standardizing Structural Causal Models](https://mlsanity.com/paper/2406.11601/) (2024) - [All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling](https://mlsanity.com/paper/2410.23501/) (2024) - [Is machine learning good or bad for the natural sciences?](https://mlsanity.com/paper/2405.18095/) (2024) - [Equivariant geometric convolutions for emulation of dynamical systems](https://mlsanity.com/paper/2305.12585/) (2023) - [Tensor learning with orthogonal, Lorentz, and symplectic symmetries](https://mlsanity.com/paper/2406.01552/) (2024) - [Equivariant Machine Learning on Graphs with Nonlinear Spectral Filters](https://mlsanity.com/paper/2406.01249/) (2024) - [Towards Coordinate- and Dimension-Agnostic Machine Learning for Partial Differential Equations](https://mlsanity.com/paper/2505.16549/) (2025) - [Decomposition of Equivariant Maps via Invariant Maps: Application to Universal Approximation under Symmetry](https://mlsanity.com/paper/2409.16922/) (2024) - [Causal Models for Growing Networks](https://mlsanity.com/paper/2504.01012/) (2025) - [If you can distinguish, you can express: Galois theory, Stone--Weierstrass, machine learning, and linguistics](https://mlsanity.com/paper/2510.09902/) (2025) ## Abstract (arXiv) Any representation of data involves arbitrary investigator choices. Because those choices are external to the data-generating process, each choice leads to an exact symmetry, corresponding to the group of transformations that takes one possible representation to another. These are the passive symmetries; they include coordinate freedom, gauge symmetry, and units covariance, all of which have led to important results in physics. In machine learning, the most visible passive symmetry is the relabeling or permutation symmetry of graphs. Our goal is to understand the implications for machine learning of the many passive symmetries in play. We discuss dos and don'ts for machine learning practice if passive symmetries are to be respected. We discuss links to causal modeling, and argue that the implementation of passive symmetries is particularly valuable when the goal of the learning problem is to generalize out of sample. This paper is conceptual: It translates among the languages of physics, mathematics, and machine-learning. We believe that consideration and implementation of passive symmetries might help machine learning in the same ways that it transformed physics in the twentieth century.