Digital implementations of deep feature extractors are intrinsically informative
Not on the map. Why?
Abstract (arXiv)
Rapid information (energy) propagation in deep feature extractors is crucial to balance computational complexity versus expressiveness as a representation of the input. We prove an upper bound for the speed of energy propagation in a unified framework that covers different neural network models, both over Euclidean and non-Euclidean domains. Additional structural information about the signal domain can be used to explicitly determine or improve the rate of decay. To illustrate this, we show global exponential energy decay for a range of 1) feature extractors with discrete-domain input signals, and 2) convolutional neural networks (CNNs) via scattering over locally compact abelian (LCA) groups.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 20 references that are in the map. First posted on arXiv 20 Feb 2025.
Builds on
9 of 20 references.
References · 9
- 2016Understanding Deep Convolutional Networkscited by 76
- 2019Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizerscited by 61
- 2015A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extractioncited by 46
- 2018Graph Convolutional Neural Networks via Scatteringcited by 46
- 2022Geometric Scattering on Measure Spacescited by 17
- 2019Understanding Graph Neural Networks with Generalized Geometric Scattering Transformscited by 12
- 2017Energy Propagation in Deep Convolutional Neural Networkscited by 5
- 2017Topology Reduction in Deep Convolutional Feature Extraction Networkscited by 3
- 2024Energy Propagation in Scattering Convolution Networks Can Be Arbitrarily Slowcited by 1
Built on by
Not cited yet.