# Digital implementations of deep feature extractors are intrinsically informative

arXiv: https://arxiv.org/abs/2502.15004 · HTML: https://mlsanity.com/paper/2502.15004/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 20 references that are in the map.
- First posted on arXiv 20 Feb 2025.

## Builds on (most cited first, 9 of 9)

- [Understanding Deep Convolutional Networks](https://mlsanity.com/paper/1601.04920/) (2016)
- [Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers](https://mlsanity.com/paper/1910.05505/) (2019)
- [A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction](https://mlsanity.com/paper/1512.06293/) (2015)
- [Graph Convolutional Neural Networks via Scattering](https://mlsanity.com/paper/1804.00099/) (2018)
- [Geometric Scattering on Measure Spaces](https://mlsanity.com/paper/2208.08561/) (2022)
- [Understanding Graph Neural Networks with Generalized Geometric Scattering Transforms](https://mlsanity.com/paper/1911.06253/) (2019)
- [Energy Propagation in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1704.03636/) (2017)
- [Topology Reduction in Deep Convolutional Feature Extraction Networks](https://mlsanity.com/paper/1707.02711/) (2017)
- [Energy Propagation in Scattering Convolution Networks Can Be Arbitrarily Slow](https://mlsanity.com/paper/2406.05121/) (2024)

## Abstract (arXiv)

Rapid information (energy) propagation in deep feature extractors is crucial to balance computational complexity versus expressiveness as a representation of the input. We prove an upper bound for the speed of energy propagation in a unified framework that covers different neural network models, both over Euclidean and non-Euclidean domains. Additional structural information about the signal domain can be used to explicitly determine or improve the rate of decay. To illustrate this, we show global exponential energy decay for a range of 1) feature extractors with discrete-domain input signals, and 2) convolutional neural networks (CNNs) via scattering over locally compact abelian (LCA) groups.