mlsanityData to 14 Oct 2026

Towards fully covariant machine learning

arXiv 2301.13724 ↗First posted 31 Jan 2023stat.ML astro-ph.IM cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Any representation of data involves arbitrary investigator choices. Because those choices are external to the data-generating process, each choice leads to an exact symmetry, corresponding to the group of transformations that takes one possible representation to another. These are the passive symmetries; they include coordinate freedom, gauge symmetry, and units covariance, all of which have led to important results in physics. In machine learning, the most visible passive symmetry is the relabeling or permutation symmetry of graphs. Our goal is to understand the implications for machine learning of the many passive symmetries in play. We discuss dos and don'ts for machine learning practice if passive symmetries are to be respected. We discuss links to causal modeling, and argue that the implementation of passive symmetries is particularly valuable when the goal of the learning problem is to generalize out of sample. This paper is conceptual: It translates among the languages of physics, mathematics, and machine-learning. We believe that consideration and implementation of passive symmetries might help machine learning in the same ways that it transformed physics in the twentieth century.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 41 of its 93 references that are in the map. First posted on arXiv 31 Jan 2023.

Builds on

41 of 93 references.

References · 41

  1. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  2. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  3. 2016Group Equivariant Convolutional Networkscited by 1,165
  4. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,102
  5. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  6. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 754
  7. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 746
  8. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 708
  9. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
  10. 2019Benign Overfitting in Linear Regressioncited by 549
  11. 2017Sharp Minima Can Generalize For Deep Netscited by 466
  12. 2018Invariant and Equivariant Graph Networkscited by 346
29 more
  1. 2018On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groupscited by 300
  2. 2022Learning Local Equivariant Representations for Large-Scale Atomistic Dynamicscited by 256
  3. 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
  4. 2021A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscited by 187
  5. 2022e3nn: Euclidean Neural Networkscited by 174
  6. 2022EquiBind: Geometric Deep Learning for Drug Binding Structure Predictioncited by 144
  7. 2021Scalars are universal: Equivariant machine learning, structured like classical physicscited by 127
  8. 2022Equivariance with Learned Canonicalization Functionscited by 109
  9. 2019Hamiltonian Graph Networks with ODE Integratorscited by 108
  10. 2020ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICAcited by 102
  11. 2022Approximately Equivariant Networks for Imperfectly Symmetric Dynamicscited by 97
  12. 2021Provably Strict Generalisation Benefit for Equivariant Modelscited by 91
  13. 2021Learning with invariances in random features and kernel modelscited by 89
  14. 2020Relative Flatness and Generalizationcited by 88
  15. 2021Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifoldscited by 73
  16. 2022Function Classes for Identifiable Nonlinear Independent Component Analysiscited by 53
  17. 2018N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentialscited by 42
  18. 2022Low Dimensional Invariant Embeddings for Universal Geometric Learningcited by 38
  19. 2021Provably Strict Generalisation Benefit for Invariance in Kernel Methodscited by 30
  20. 2022Dimensionally Consistent Learning with Buckingham Picited by 22
  21. 2021Implicit Bias of Linear Equivariant Networkscited by 19
  22. 2022Group-invariant max filteringcited by 18
  23. 2022Dimensionless machine learning: Imposing exact units equivariancecited by 17
  24. 2019Experimental performance of graph neural networks on random instances of max-cutcited by 15
  25. 2020Dimensionality reduction, regularization, and generalization in overparameterized regressionscited by 13
  26. 2022Machine learning and invariant theorycited by 13
  27. 2019Covariance in Physics and Convolutional Neural Networkscited by 12
  28. 2021A simple equivariant machine learning method for dynamics based on scalarscited by 6
  29. 2021Fitting very flexible models: Linear regression with large numbers of parameterscited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 6; 2025: 3.

10 papers.

  1. 2024Standardizing Structural Causal Modelscited by 17
  2. 2024All or None: Identifiable Linear Properties of Next-token Predictors in Language Modelingcited by 14
  3. 2024Is machine learning good or bad for the natural sciences?cited by 4
  4. 2023Equivariant geometric convolutions for emulation of dynamical systemscited by 3
  5. 2024Tensor learning with orthogonal, Lorentz, and symplectic symmetriescited by 3
  6. 2024Equivariant Machine Learning on Graphs with Nonlinear Spectral Filterscited by 2
  7. 2025Towards Coordinate- and Dimension-Agnostic Machine Learning for Partial Differential Equationscited by 2
  8. 2024Decomposition of Equivariant Maps via Invariant Maps: Application to Universal Approximation under Symmetrycited by 0
  9. 2025Causal Models for Growing Networkscited by 0
  10. 2025If you can distinguish, you can express: Galois theory, Stone--Weierstrass, machine learning, and linguisticscited by 0