# Continuous Generative Neural Networks: A Wavelet-Based Architecture in Function Spaces

arXiv: https://arxiv.org/abs/2205.14627 · HTML: https://mlsanity.com/paper/2205.14627/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 90 references that are in the map.
- First posted on arXiv 29 May 2022.

## Builds on (most cited first, 30 of 33)

- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [Invertible Residual Networks](https://mlsanity.com/paper/1811.00995/) (2018)
- [Compressed Sensing using Generative Models](https://mlsanity.com/paper/1703.03208/) (2017)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Analyzing Inverse Problems with Invertible Neural Networks](https://mlsanity.com/paper/1808.04730/) (2018)
- [Adversarial Generation of Continuous Images](https://mlsanity.com/paper/2011.12026/) (2020)
- [An Introduction to Deep Generative Modeling](https://mlsanity.com/paper/2103.05180/) (2021)
- [Deep learning for undersampled MRI reconstruction](https://mlsanity.com/paper/1709.02576/) (2017)
- [Generative Models as Distributions of Functions](https://mlsanity.com/paper/2102.04776/) (2021)
- [Deep Scattering Spectrum](https://mlsanity.com/paper/1304.6763/) (2013)
- [Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis](https://mlsanity.com/paper/1701.05927/) (2017)
- [Blind Image Deconvolution using Deep Generative Priors](https://mlsanity.com/paper/1802.04073/) (2018)
- [Trumpets: Injective Flows for Inference and Inverse Problems](https://mlsanity.com/paper/2102.10461/) (2021)
- [Inverting Deep Generative models, One layer at a time](https://mlsanity.com/paper/1906.07437/) (2019)
- [Stochastic Normalizing Flows for Inverse Problems: a Markov Chains Viewpoint](https://mlsanity.com/paper/2109.11375/) (2021)
- [Stabilizing Invertible Neural Networks Using Mixture Models](https://mlsanity.com/paper/2009.02994/) (2020)
- [Globally Injective ReLU Networks](https://mlsanity.com/paper/2006.08464/) (2020)
- [Deep Learning-Based Solvability of Underdetermined Inverse Problems in Medical Imaging](https://mlsanity.com/paper/2001.01432/) (2020)
- [A Generative Variational Model for Inverse Problems in Imaging](https://mlsanity.com/paper/2104.12630/) (2021)
- [Universal Joint Approximation of Manifolds and Densities by Simple Injective Flows](https://mlsanity.com/paper/2110.04227/) (2021)
- [iUNets: Fully invertible U-Nets with Learnable Up- and Downsampling](https://mlsanity.com/paper/2005.05220/) (2020)

## Built on by (most cited first, 6 of 6)

- [Representation Equivalent Neural Operators: a Framework for Alias-free Operator Learning](https://mlsanity.com/paper/2305.19913/) (2023)
- [Multilevel Diffusion: Infinite Dimensional Score-Based Diffusion Models for Image Generation](https://mlsanity.com/paper/2303.04772/) (2023)
- [Manifold Learning by Mixture Models of VAEs for Inverse Problems](https://mlsanity.com/paper/2303.15244/) (2023)
- [Globally injective and bijective neural operators](https://mlsanity.com/paper/2306.03982/) (2023)
- [Can neural operators always be continuously discretized?](https://mlsanity.com/paper/2412.03393/) (2024)
- [Infinite dimensional generative sensing](https://mlsanity.com/paper/2603.03196/) (2026)

## Abstract (arXiv)

In this work, we present and study Continuous Generative Neural Networks (CGNNs), namely, generative models in the continuous setting: the output of a CGNN belongs to an infinite-dimensional function space. The architecture is inspired by DCGAN, with one fully connected layer, several convolutional layers and nonlinear activation functions. In the continuous L2 setting, the dimensions of the spaces of each layer are replaced by the scales of a multiresolution analysis of a compactly supported wavelet. We present conditions on the convolutional filters and on the nonlinearity that guarantee that a CGNN is injective. This theory finds applications to inverse problems, and allows for deriving Lipschitz stability estimates for (possibly nonlinear) infinite-dimensional inverse problems with unknowns belonging to the manifold generated by a CGNN. Several numerical simulations, including signal deblurring, illustrate and validate this approach.