mlsanityData to 7 Oct 2026

Emergent Equivariance in Deep Ensembles

arXiv 2403.03103 ↗First posted 5 Mar 2024cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

We show that deep ensembles become equivariant for all inputs and at all training times by simply using data augmentation. Crucially, equivariance holds off-manifold and for any architecture in the infinite width limit. The equivariance is emergent in the sense that predictions of individual ensemble members are not equivariant but their collective prediction is. Neural tangent kernel theory is used to derive this result and we verify our theoretical insights using detailed numerical experiments.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 38 of its 43 references that are in the map. First posted on arXiv 5 Mar 2024.

Builds on

38 of 43 references.

References · 38

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  3. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  4. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  5. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  6. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  7. 2017Deep Neural Networks as Gaussian Processescited by 617
  8. 2019On Exact Computation with an Infinitely Wide Neural Netcited by 572
  9. 2019General E(2)-Equivariant Steerable CNNscited by 441
  10. 2021Ensemble deep learning: A reviewcited by 283
  11. 2020Feature Learning in Infinite-Width Neural Networkscited by 182
  12. 2021Frame Averaging for Invariant and Equivariant Network Designcited by 166
26 more
  1. 2019Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernelscited by 160
  2. 2019Neural Tangents: Fast and Easy Infinite Neural Networks in Pythoncited by 149
  3. 2019Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivationcited by 146
  4. 2018A Kernel Theory of Modern Data Augmentationcited by 106
  5. 2022Equivariance with Learned Canonicalization Functionscited by 106
  6. 2019Dynamics of Deep Neural Networks and Neural Tangent Hierarchycited by 96
  7. 2018Roto-Translation Covariant Convolutional Networks for Medical Image Analysiscited by 91
  8. 2021Provably Strict Generalisation Benefit for Equivariant Modelscited by 90
  9. 2021Learning with invariances in random features and kernel modelscited by 89
  10. 2019Non-Gaussian processes and neural networks at finite widthscited by 78
  11. 2019Enhanced Convolutional Neural Tangent Kernelscited by 76
  12. 2021Geometric Deep Learning and Equivariant Neural Networkscited by 64
  13. 2020Neural Networks and Quantum Field Theorycited by 58
  14. 2023Equivariant Adaptation of Large Pretrained Modelscited by 40
  15. 2019Improved Generalization Bounds of Group Invariant / Equivariant Deep Networks via Quotient Feature Spacescited by 34
  16. 2021Nonperturbative renormalization for the neural network-QFT correspondencecited by 24
  17. 2022Equivariance versus Augmentation for Spherical Imagescited by 24
  18. 2022Neural Tangent Kernel: A Surveycited by 22
  19. 2022Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Modelscited by 22
  20. 2021A Neural Tangent Kernel Perspective of GANscited by 21
  21. 2023Optimization Dynamics of Equivariant and Augmented Neural Networkscited by 17
  22. 2022Few-shot Backdoor Attacks via Neural Tangent Kernelscited by 15
  23. 2015Learning with Group Invariant Features: A Kernel Perspectivecited by 13
  24. 2016Local Group Invariant Representations via Orbit Embeddingscited by 12
  25. 2022On the Neural Tangent Kernel Analysis of Randomly Pruned Neural Networkscited by 9
  26. 2020Hierarchical Deep Learning Ensemble to Automate the Classification of Breast Cancer Pathology Reports by ICD-O Topographycited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 8; 2026: 6.

18 papers.

Sort:

By area:

  1. 2024Orb: A Fast, Scalable Neural Network Potentialcited by 63
  2. 2024On the Ability of Deep Networks to Learn Symmetries from Data: A Neural Kernel Theorycited by 10
  3. 2025How simple can you go? An off-the-shelf transformer approach to molecular dynamicscited by 10
  4. 2024Ensembles provably learn equivariance through data augmentationcited by 8
  5. 2024Equivariant Neural Tangent Kernelscited by 4
  6. 2025Data Augmentation and Regularization for Learning Group Equivariancecited by 3
  7. 2025Understanding Learning Invariance in Deep Linear Networkscited by 3
  8. 2025Not All Explanations for Deep Learning Phenomena Are Equally Valuablecited by 3
  9. 2025Finite-Width Neural Tangent Kernels from Feynman Diagramscited by 3
  10. 2025Do we need equivariant models for molecule generation?cited by 2
  11. 2025How Ensembles of Distilled Policies Improve Generalisation in Reinforcement Learningcited by 1
  12. 2025Implicit Bias and Invariance: How Hopfield Networks Efficiently Learn Graph Orbitscited by 0
6 more
  1. 2026Latent Equivariant Operators for Robust Object Recognition: Promises and Challengescited by 0
  2. 2026On the Equivariant Learning of the Q-tensor Order Parametercited by 0
  3. 2026Equivariance and Augmentation for Bayesian Neural Networkscited by 0
  4. 2026Generalization in offline RL: The structure is more important than the amount of pessimismcited by 0
  5. 2026Boosting Data Augmentation with Stochastic Weight Averagingcited by 0
  6. 2026Equivariant Covariance Tensors: Guaranteed SPD Uncertainty for Tensor-Valued Geometric Learningcited by 0