mlsanityData to 7 Oct 2026

Deep Neural Networks with Efficient Guaranteed Invariances

arXiv 2303.01567 ↗First posted 2 Mar 2023cs.LG cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

We address the problem of improving the performance and in particular the sample complexity of deep neural networks by enforcing and guaranteeing invariances to symmetry transformations rather than learning them from data. Group-equivariant convolutions are a popular approach to obtain equivariant representations. The desired corresponding invariance is then imposed using pooling operations. For rotations, it has been shown that using invariant integration instead of pooling further improves the sample complexity. In this contribution, we first expand invariant integration beyond rotations to flips and scale transformations. We then address the problem of incorporating multiple desired invariances into a single network. For this purpose, we propose a multi-stream architecture, where each stream is invariant to a different transformation such that the network can simultaneously benefit from multiple invariances. We demonstrate our approach with successful experiments on Scaled-MNIST, SVHN, CIFAR-10 and STL-10.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 53 of its 78 references that are in the map. First posted on arXiv 2 Mar 2023.

Builds on

53 of 78 references.

References · 53

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2016Wide Residual Networkscited by 3,057
  4. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  5. 2017Deformable Convolutional Networkscited by 1,481
  6. 2015Spatial Transformer Networkscited by 1,423
  7. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  8. 2016Group Equivariant Convolutional Networkscited by 1,154
  9. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
  10. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
  11. 2018Spherical CNNscited by 452
  12. 2019General E(2)-Equivariant Steerable CNNscited by 441
38 more
  1. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 348
  2. 2018On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groupscited by 296
  3. 2016Harmonic Networks: Deep Translation and Rotation Equivariancecited by 293
  4. 2019Gauge Equivariant Convolutional Networks and the Icosahedral CNNcited by 262
  5. 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
  6. 2019Fast AutoAugmentcited by 243
  7. 2018A General Theory of Equivariant CNNs on Homogeneous Spacescited by 223
  8. 2017Learning Steerable Filters for Rotation Equivariant CNNscited by 193
  9. 2021A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscited by 185
  10. 2018Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Networkcited by 168
  11. 2021Frame Averaging for Invariant and Equivariant Network Designcited by 166
  12. 2019Deep Scale-spaces: Equivariance Over Scalecited by 113
  13. 2019Scale-Equivariant Steerable Networkscited by 110
  14. 2020LieTransformer: Equivariant self-attention for Lie Groupscited by 100
  15. 2016Rotation equivariant vector field networkscited by 98
  16. 2019B-Spline CNNs on Lie Groupscited by 97
  17. 2018Roto-Translation Covariant Convolutional Networks for Medical Image Analysiscited by 91
  18. 2021Provably Strict Generalisation Benefit for Equivariant Modelscited by 90
  19. 2017Polar Transformer Networkscited by 81
  20. 2019Spherical CNNs on Unstructured Gridscited by 76
  21. 2020DeepSphere: a graph-based spherical CNNcited by 65
  22. 2020Group Equivariant Stand-Alone Self-Attention For Visioncited by 63
  23. 2020Attentive Group Equivariant Convolutional Networkscited by 62
  24. 2018CubeNet: Equivariance to 3D Rotation and Translationcited by 59
  25. 2020Spin-Weighted Spherical CNNscited by 50
  26. 2014Scale-Invariant Convolutional Neural Networkscited by 40
  27. 2021DISCO: accurate Discrete Scale Convolutionscited by 35
  28. 2014Locally Scale-Invariant Convolutional Neural Networkscited by 34
  29. 2018Scattering Networks for Hybrid Representation Learningcited by 33
  30. 2018HexaConvcited by 27
  31. 2020Theoretical Aspects of Group Equivariant Neural Networkscited by 25
  32. 2019Scale Steerable Filters for Locally Scale-Invariant Convolutional Neural Networkscited by 22
  33. 2018Scale equivariance in CNNs with vector fieldscited by 20
  34. 2020GraN: An Efficient Gradient-Norm Based Detector for Adversarial and Misclassified Examplescited by 18
  35. 2021Equivariant Networks for Pixelized Spherescited by 16
  36. 2021Group Equivariant Subsamplingcited by 15
  37. 2019Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring In Datacited by 10
  38. 2021Iterative SE(3)-Transformerscited by 10

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 2.

2 papers.

  1. 2023From Bricks to Bridges: Product of Invariances to Enhance Latent Space Communicationcited by 20
  2. 2023GIT: Detecting Uncertainty, Out-Of-Distribution and Adversarial Samples using Gradients and Invariance Transformationscited by 0