Invariance Measures for Neural Networks
Not on the map. Why?
Abstract (arXiv)
Invariances in neural networks are useful and necessary for many tasks. However, the representation of the invariance of most neural network models has not been characterized. We propose measures to quantify the invariance of neural networks in terms of their internal representation. The measures are efficient and interpretable, and can be applied to any neural network model. They are also more sensitive to invariance than previously defined measures. We validate the measures and their properties in the domain of affine transformations and the CIFAR10 and MNIST datasets, including their stability and interpretability. Using the measures, we perform a first analysis of CNN models and show that their internal invariance is remarkably stable to random weight initializations, but not to changes in dataset or transformation. We believe the measures will enable new avenues of research in invariance representation.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 23 of its 46 references that are in the map. First posted on arXiv 26 Oct 2023.
Builds on
23 of 46 references.
References · 23
- 2013Visualizing and Understanding Convolutional Networkscited by 2,278
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
- 2017Self-Normalizing Neural Networkscited by 591
- 2014Understanding image representations by measuring their equivariance and equivalencecited by 240
- 2018Why do deep convolutional networks generalize so poorly to small image transformations?cited by 198
- 2017Controllable Invariance through Adversarial Feature Learningcited by 127
- 2016Exploiting Cyclic Symmetry in Convolutional Neural Networkscited by 91
- 2016Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Unitscited by 86
- 2017Exploring the Landscape of Spatial Robustnesscited by 81
- 2020Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbationscited by 55
- 2017NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Mapscited by 39
11 more
- 2017Quantifying Translation-Invariance in Convolutional Neural Networkscited by 30
- 2015Manitest: Are classifiers really invariant?cited by 27
- 2023Revisiting Data Augmentation for Rotational Invariance in Convolutional Neural Networkscited by 16
- 2019Sample-Optimal Low-Rank Approximation of Distance Matricescited by 15
- 2017Towards Understanding the Invertibility of Convolutional Neural Networkscited by 10
- 2018Diverse feature visualizations reveal invariances in early layers of deep neural networkscited by 8
- 2017Explaining the Unexplained: A CLass-Enhanced Attentive Response (CLEAR) Approach to Understanding Deep Neural Networkscited by 4
- 2018Studying Invariances of Trained Convolutional Neural Networkscited by 2
- 2014Learning Deep Object Detectors from 3D Modelscited by 1
- 20183D Human Pose Estimation with Siamese Equivariant Embeddingcited by 1
- 2018The Effect of Learning Strategy versus Inherent Architecture Properties on the Ability of Convolutional Neural Networks to Develop Transformation Invariancecited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.