# Equivariance and Augmentation for Bayesian Neural Networks

arXiv: https://arxiv.org/abs/2606.26273 · HTML: https://mlsanity.com/paper/2606.26273/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 36 references that are in the map.
- First posted on arXiv 24 Jun 2026.

## Builds on (most cited first, 26 of 26)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [Universal Invariant and Equivariant Graph Neural Networks](https://mlsanity.com/paper/1905.04943/) (2019)
- [Frame Averaging for Invariant and Equivariant Network Design](https://mlsanity.com/paper/2110.03336/) (2021)
- [Scalars are universal: Equivariant machine learning, structured like classical physics](https://mlsanity.com/paper/2106.06610/) (2021)
- [Approximately Equivariant Networks for Imperfectly Symmetric Dynamics](https://mlsanity.com/paper/2201.11969/) (2022)
- [Provably Strict Generalisation Benefit for Equivariant Models](https://mlsanity.com/paper/2102.10333/) (2021)
- [On the Benefits of Invariance in Neural Networks](https://mlsanity.com/paper/2005.00178/) (2020)
- [Geometric Deep Learning and Equivariant Neural Networks](https://mlsanity.com/paper/2105.13926/) (2021)
- [Swallowing the Bitter Pill: Simplified Scalable Conformer Generation](https://mlsanity.com/paper/2311.17932/) (2023)
- [Does equivariance matter at scale?](https://mlsanity.com/paper/2410.23179/) (2024)
- [Equivariance versus Augmentation for Spherical Images](https://mlsanity.com/paper/2202.03990/) (2022)
- [Implicit Bias of Linear Equivariant Networks](https://mlsanity.com/paper/2110.06084/) (2021)
- [Optimization Dynamics of Equivariant and Augmented Neural Networks](https://mlsanity.com/paper/2303.13458/) (2023)
- [Emergent Equivariance in Deep Ensembles](https://mlsanity.com/paper/2403.03103/) (2024)
- [Learning Invariant Weights in Neural Networks](https://mlsanity.com/paper/2202.12439/) (2022)
- [Group invariant machine learning by fundamental domain projections](https://mlsanity.com/paper/2202.02164/) (2022)
- [Training or Architecture? How to Incorporate Invariance in Neural Networks](https://mlsanity.com/paper/2106.10044/) (2021)
- [Ensembles provably learn equivariance through data augmentation](https://mlsanity.com/paper/2410.01452/) (2024)
- [On the Implicit Bias of Linear Equivariant Steerable Networks](https://mlsanity.com/paper/2303.04198/) (2023)
- [Data Augmentation and Regularization for Learning Group Equivariance](https://mlsanity.com/paper/2502.06547/) (2025)
- [Understanding Learning Invariance in Deep Linear Networks](https://mlsanity.com/paper/2506.13714/) (2025)
- [A Bayesian Approach to Invariant Deep Neural Networks](https://mlsanity.com/paper/2107.09301/) (2021)

## Abstract (arXiv)

Symmetries are important for many deep learning tasks, ranging from applications in the sciences to medical imaging. However, there is an ongoing debate about whether to impose symmetry constraints on the neural network architecture (yielding equivariant neural networks) or learn them from augmented training data. Although equivariant networks are well-studied theoretically, much less is known about data augmentation, since analyzing augmentation requires control over the training dynamics. Inspired by recent results that show that augmented infinite deep ensembles are exactly equivariant, we study data augmentation for Bayesian neural networks (BNNs) trained with variational inference. We focus on variational distributions in the exponential family and derive conditions under which exact equivariance is reached. We furthermore obtain bounds on the equivariance error and introduce three novel symmetrization techniques which boost the effect of data augmentation in this setting. We conduct extensive numerical experiments which show that one of our symmetrization methods (orbit expansion) outperforms the baseline in both equivariance and overall performance. Our code is available at github.com/dmw1998/augment-BNNs