# Generalization capabilities of translationally equivariant neural networks

arXiv: https://arxiv.org/abs/2103.14686 · HTML: https://mlsanity.com/paper/2103.14686/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 26 Mar 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [Rotation Equivariant CNNs for Digital Pathology](https://mlsanity.com/paper/1806.03962/) (2018)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [Steerable CNNs](https://mlsanity.com/paper/1612.08498/) (2016)
- [Harmonic Networks: Deep Translation and Rotation Equivariance](https://mlsanity.com/paper/1612.04642/) (2016)
- [Gauge Equivariant Convolutional Networks and the Icosahedral CNN](https://mlsanity.com/paper/1902.04615/) (2019)
- [Equivariant flow-based sampling for lattice gauge theory](https://mlsanity.com/paper/2003.06413/) (2020)
- [Sampling using SU(N) gauge equivariant flows](https://mlsanity.com/paper/2008.05456/) (2020)
- [CubeNet: Equivariance to 3D Rotation and Translation](https://mlsanity.com/paper/1804.04458/) (2018)
- [Lattice gauge equivariant convolutional neural networks](https://mlsanity.com/paper/2012.12901/) (2020)
- [CNN-based Density Estimation and Crowd Counting: A Survey](https://mlsanity.com/paper/2003.12783/) (2020)
- [Roto-Translation Equivariant Convolutional Networks: Application to Histopathology Image Analysis](https://mlsanity.com/paper/2002.08725/) (2020)
- [A rotation-equivariant convolutional neural network model of primary visual cortex](https://mlsanity.com/paper/1809.10504/) (2018)
- [Towards Novel Insights in Lattice Field Theory with Explainable Machine Learning](https://mlsanity.com/paper/2003.01504/) (2020)
- [Extending machine learning classification capabilities with histogram reweighting](https://mlsanity.com/paper/2004.14341/) (2020)

## Built on by (most cited first, 18 of 18)

- [Geometry-Complete Diffusion for 3D Molecule Generation and Optimization](https://mlsanity.com/paper/2302.04313/) (2023)
- [Implicit Bias of Linear Equivariant Networks](https://mlsanity.com/paper/2110.06084/) (2021)
- [Applications of Machine Learning to Lattice Quantum Field Theory](https://mlsanity.com/paper/2202.05838/) (2022)
- [How Tempering Fixes Data Augmentation in Bayesian Neural Networks](https://mlsanity.com/paper/2205.13900/) (2022)
- [Machine Learning of Thermodynamic Observables in the Presence of Mode Collapse](https://mlsanity.com/paper/2111.11303/) (2021)
- [Snowmass 2021 Computational Frontier CompF03 Topical Group Report: Machine Learning](https://mlsanity.com/paper/2209.07559/) (2022)
- [Approximately Equivariant Neural Processes](https://mlsanity.com/paper/2406.13488/) (2024)
- [Geometrical aspects of lattice gauge equivariant convolutional neural networks](https://mlsanity.com/paper/2303.11448/) (2023)
- [mathbbZ2× mathbbZ2 Equivariant Quantum Neural Networks: Benchmarking against Classical Neural Networks](https://mlsanity.com/paper/2311.18744/) (2023)
- [EGR: Equivariant Graph Refinement and Assessment of 3D Protein Complex Structures](https://mlsanity.com/paper/2205.10390/) (2022)
- [Gauge-Equivariant Graph Neural Networks for Lattice Gauge Theories](https://mlsanity.com/paper/2604.20797/) (2026)
- [Graph Neural Networks in the Wilson Loop Representation of Abelian Lattice Gauge Theories](https://mlsanity.com/paper/2605.03901/) (2026)
- [Prediction and compression of lattice QCD data using machine learning algorithms on quantum annealer](https://mlsanity.com/paper/2112.02120/) (2021)
- [Lattice gauge symmetry in neural networks](https://mlsanity.com/paper/2111.04389/) (2021)
- [Generalization capabilities of neural networks in lattice applications](https://mlsanity.com/paper/2112.12474/) (2021)
- [Invariant multiscale neural networks for data-scarce scientific applications](https://mlsanity.com/paper/2406.08318/) (2024)
- [Revisiting Neural Processes via Fourier Transform and Volterra Series](https://mlsanity.com/paper/2606.01172/) (2026)
- [Learning Macroscopic Dynamics without Reconstructing Microscopic States](https://mlsanity.com/paper/2609.37392/) (2026)

## Abstract (arXiv)

The rising adoption of machine learning in high energy physics and lattice field theory necessitates the re-evaluation of common methods that are widely used in computer vision, which, when applied to problems in physics, can lead to significant drawbacks in terms of performance and generalizability. One particular example for this is the use of neural network architectures that do not reflect the underlying symmetries of the given physical problem. In this work, we focus on complex scalar field theory on a two-dimensional lattice and investigate the benefits of using group equivariant convolutional neural network architectures based on the translation group. For a meaningful comparison, we conduct a systematic search for equivariant and non-equivariant neural network architectures and apply them to various regression and classification tasks. We demonstrate that in most of these tasks our best equivariant architectures can perform and generalize significantly better than their non-equivariant counterparts, which applies not only to physical parameters beyond those represented in the training set, but also to different lattice sizes.