# On Universal Approximation by Neural Networks with Uniform Guarantees on Approximation of Infinite Dimensional Maps

arXiv: https://arxiv.org/abs/1910.01545 · HTML: https://mlsanity.com/paper/1910.01545/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Oct 2019.

## Built on by (most cited first, 8 of 8)

- [Generic bounds on the approximation error for physics-informed (and) operator learning](https://mlsanity.com/paper/2205.11393/) (2022)
- [Category Theory in Machine Learning](https://mlsanity.com/paper/2106.07032/) (2021)
- [Deep Learning for Functional Data Analysis with Adaptive Basis Layers](https://mlsanity.com/paper/2106.10414/) (2021)
- [AI Descartes: Combining Data and Theory for Derivable Scientific Discovery](https://mlsanity.com/paper/2109.01634/) (2021)
- [Neural Operator: Is data all you need to model the world? An insight into the paradigm of data-driven scientific ML](https://mlsanity.com/paper/2301.13331/) (2023)
- [Shape-Informed Clustering of Multi-Dimensional Functional Data via Deep Functional Autoencoders](https://mlsanity.com/paper/2509.22969/) (2025)
- [Neural networks in non-metric spaces](https://mlsanity.com/paper/2406.09310/) (2024)
- [Convex Distillation: Efficient Compression of Deep Networks via Convex Optimization](https://mlsanity.com/paper/2410.06567/) (2024)

## Abstract (arXiv)

The study of universal approximation of arbitrary functions f: X → Y by neural networks has a rich and thorough history dating back to Kolmogorov (1957). In the case of learning finite dimensional maps, many authors have shown various forms of the universality of both fixed depth and fixed width neural networks. However, in many cases, these classical results fail to extend to the recent use of approximations of neural networks with infinitely many units for functional data analysis, dynamical systems identification, and other applications where either X or Y become infinite dimensional. Two questions naturally arise: which infinite dimensional analogues of neural networks are sufficient to approximate any map f: X → Y, and when do the finite approximations to these analogues used in practice approximate f uniformly over its infinite dimensional domain X? In this paper, we answer the open question of universal approximation of nonlinear operators when X and Y are both infinite dimensional. We show that for a large class of different infinite analogues of neural networks, any continuous map can be approximated arbitrarily closely with some mild topological conditions on X. Additionally, we provide the first lower-bound on the minimal number of input and output units required by a finite approximation to an infinite neural network to guarantee that it can uniformly approximate any nonlinear operator using samples from its inputs and outputs.