mlsanityData to 7 Oct 2026

Learning Invariances in Neural Networks

arXiv 2010.11882 ↗First posted 22 Oct 2020cs.LG stat.MLcited by 52

Not on the map. Why?

Abstract (arXiv)

Invariances to translations have imbued convolutional neural networks with powerful generalization properties. However, we often do not know a priori what invariances are present in the data, or to what extent a model should be invariant to a given symmetry group. We show how to learn invariances and equivariances by parameterizing a distribution over augmentations and optimizing the training loss simultaneously with respect to the network parameters and augmentation parameters. With this simple procedure we can recover the correct set and extent of invariances on image classification, regression, segmentation, and molecular property prediction from a large space of augmentations, on training data alone.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 14; 2023: 8; 2024: 7; 2025: 4; 2026: 1.

52 papers.

Sort:

By area:

  1. 2021Automatic Symmetry Discovery with Lie Algebra Convolutional Networkcited by 82
  2. 2021Residual Pathway Priors for Soft Equivariance Constraintscited by 74
  3. 2024Orb: A Fast, Scalable Neural Network Potentialcited by 63
  4. 2022Learning Symmetric Embeddings for Equivariant World Modelscited by 54
  5. 2022Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximationscited by 42
  6. 2022Relaxing Equivariance Constraints with Non-stationary Continuous Filterscited by 41
  7. 2021Data augmentation in Bayesian neural networks and the cold posterior effectcited by 36
  8. 2021Equitable modelling of brain imaging by counterfactual augmentation with morphologically constrained 3D deep generative modelscited by 36
  9. 2021An Empirical Framework for Domain Generalization in Clinical Settingscited by 32
  10. 2022Sample Efficiency of Data Augmentation Consistency Regularizationcited by 28
  11. 2021Noether Networks: Meta-Learning Useful Conserved Quantitiescited by 26
  12. 2021Grounding inductive biases in natural images:invariance stems from variations in datacited by 25
38 more
  1. 2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 23
  2. 2023Learning Layer-wise Equivariances Automatically using Gradientscited by 23
  3. 2021Understanding the Generalization Benefit of Model Invariance from a Data Perspectivecited by 22
  4. 2021Last Layer Marginal Likelihood for Invariance Learningcited by 21
  5. 2022Equivariance Discovery by Learned Parameter-Sharingcited by 18
  6. 2022Learning Invariant Weights in Neural Networkscited by 16
  7. 2022Dimensionless machine learning: Imposing exact units equivariancecited by 16
  8. 2022Learning Instance-Specific Augmentations by Capturing Local Invariancescited by 15
  9. 2021Addressing the Topological Defects of Disentanglement via Distributed Operatorscited by 14
  10. 2021Predicting Deep Neural Network Generalization with Perturbation Response Curvescited by 14
  11. 2024Symmetry Discovery Beyond Affine Transformationscited by 14
  12. 2022Do Deep Networks Transfer Invariances Across Classes?cited by 12
  13. 2023Stochastic Marginal Likelihood Gradients using Neural Tangent Kernelscited by 10
  14. 2021Machine learning a manifoldcited by 9
  15. 2023Amortised Invariance Learning for Contrastive Self-Supervisioncited by 8
  16. 2021Neural Photometry-guided Visual Attribute Transfercited by 6
  17. 2022Deep invariant networks with differentiable augmentation layerscited by 6
  18. 2023A Unified Framework for Discovering Discrete Symmetriescited by 6
  19. 2024Variational Inference Failures Under Model Symmetries: Permutation Invariant Posteriors for Bayesian Neural Networkscited by 6
  20. 2024Learning Symmetries via Weight-Sharing with Doubly Stochastic Tensorscited by 5
  21. 2023Self-Supervised Detection of Perfect and Partial Input-Dependent Symmetriescited by 4
  22. 2024Noether's razor: Learning Conserved Quantitiescited by 4
  23. 2025Towards Invariance to Node Identifiers in Graph Neural Networkscited by 4
  24. 2024A Stochastic Approach to Bi-Level Optimization for Hyperparameter Optimization and Meta Learningcited by 3
  25. 2023Learning to Transform for Generalizable Instance-wise Invariancecited by 2
  26. 2025Learning Equivariant Functions via Quadratic Formscited by 2
  27. 2021Automated and Autonomous Experiment in Electron and Scanning Probe Microscopycited by 1
  28. 2021Rotating spiders and reflecting dogs: a class conditional approach to learning data augmentation distributionscited by 1
  29. 2021A Bayesian Approach to Invariant Deep Neural Networkscited by 1
  30. 2022On the Robustness and Anomaly Detection of Sparse Neural Networkscited by 1
  31. 2024On the Utilization of Unique Node Identifiers in Graph Neural Networkscited by 1
  32. 2025RECON: Robust symmetry discovery via Explicit Canonical Orientation Normalizationcited by 1
  33. 2021Topographic VAEs learn Equivariant Capsulescited by 0
  34. 2021DNA: Dynamic Network Augmentationcited by 0
  35. 2022Warped Convolutional Networks: Bridge Homography to sl(3) algebra by Group Convolutioncited by 0
  36. 2022HyperInvariances: Amortizing Invariance Learningcited by 0
  37. 2023Learning Linear Groups in Neural Networkscited by 0
  38. 2023Randomized Dimension Reduction with Statistical Guaranteescited by 0

50 most cited of 52 All 52 (JSON)