mlsanityData to 14 Oct 2026

Deep Neural Networks as Iterated Function Systems and a Generalization Bound

arXiv 2601.19958 ↗First posted 27 Jan 2026stat.ML cs.LG math.PRcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) achieve remarkable performance on a wide range of tasks, yet their mathematical analysis remains fragmented: stability and generalization are typically studied in disparate frameworks and on a case-by-case basis. Architecturally, DNNs rely on the recursive application of parametrized functions, a mechanism that can be unstable and difficult to train, making stability a primary concern. Even when training succeeds, there are few rigorous results on how well such models generalize beyond the observed data, especially in the generative setting. In this work, we leverage the theory of stochastic Iterated Function Systems (IFS) and show that two important deep architectures can be viewed as, or canonically associated with, place-dependent IFS. This connection allows us to import results from random dynamical systems to (i) establish the existence and uniqueness of invariant measures under suitable contractivity assumptions, and (ii) derive a Wasserstein generalization bound for generative modeling. The bound naturally leads to a new training objective that directly controls the collage-type approximation error between the data distribution and its image under the learned transfer operator. We illustrate the theory on a controlled 2D example and empirically evaluate the proposed objective on standard image datasets (MNIST, CelebA, CIFAR-10).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 41 references that are in the map. First posted on arXiv 27 Jan 2026.

Builds on

22 of 41 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2018Neural Ordinary Differential Equationscited by 3,779
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  6. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  7. 2020Reformer: The Efficient Transformercited by 1,668
  8. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  9. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  10. 2020Rethinking Attention with Performerscited by 1,601
  11. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  12. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,323
10 more
  1. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 471
  2. 2023A mathematical perspective on Transformerscited by 164
  3. 2023The emergence of clusters in self-attention dynamicscited by 140
  4. 2021Sinkformers: Transformers with Doubly Stochastic Attentioncited by 138
  5. 2019Robust and interpretable blind image denoising via bias-free convolutional neural networkscited by 90
  6. 2023Wasserstein Convergence Guarantees for a General Class of Score-Based Generative Modelscited by 58
  7. 2023How Smooth Is Attention?cited by 44
  8. 2023Explicitly Minimizing the Blur Error of Variational Autoencoderscited by 31
  9. 2023Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iterationcited by 19
  10. 2022Self-Similarity Priors: Neural Collages as Differentiable Fractal Representationscited by 6

Built on by

Not cited yet.