# Neural networks in non-metric spaces

arXiv: https://arxiv.org/abs/2406.09310 · HTML: https://mlsanity.com/paper/2406.09310/

## Key facts

- Cited by 3 papers in the map (data through 7 Oct 2026).
- Builds on 15 of its 56 references that are in the map.
- First posted on arXiv 13 Jun 2024.

## Builds on (most cited first, 15 of 15)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [Universal Approximation with Deep Narrow Networks](https://mlsanity.com/paper/1905.08539/) (2019)
- [Approximating Continuous Functions by ReLU Nets of Minimal Width](https://mlsanity.com/paper/1710.11278/) (2017)
- [Echo state networks are universal](https://mlsanity.com/paper/1806.00797/) (2018)
- [An overview on deep learning-based approximation methods for partial differential equations](https://mlsanity.com/paper/2012.12348/) (2020)
- [Steps Toward Deep Kernel Methods from Infinite Neural Networks](https://mlsanity.com/paper/1508.05133/) (2015)
- [Deep neural networks, generic universal interpolation, and controlled ODEs](https://mlsanity.com/paper/1908.07838/) (2019)
- [Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer](https://mlsanity.com/paper/2201.13094/) (2022)
- [On Universal Approximation by Neural Networks with Uniform Guarantees on Approximation of Infinite Dimensional Maps](https://mlsanity.com/paper/1910.01545/) (2019)
- [Pricing options on flow forwards by neural networks in Hilbert space](https://mlsanity.com/paper/2202.11606/) (2022)

## Built on by (most cited first, 3 of 3)

- [On shallow feedforward neural networks with inputs from a topological space](https://mlsanity.com/paper/2504.02321/) (2025)
- [Universality of shallow and deep neural networks on non-Euclidean spaces](https://mlsanity.com/paper/2602.23381/) (2026)
- [Universal Approximation of Nonlinear Operators and Their Derivatives](https://mlsanity.com/paper/2605.15285/) (2026)

## Abstract (arXiv)

Leveraging the infinite dimensional neural network architecture we proposed in arXiv:2109.13512v4 and which can process inputs from Fr'echet spaces, and using the universal approximation property shown therein, we now largely extend the scope of this architecture by proving several universal approximation theorems for a vast class of input and output spaces. More precisely, the input space mathfrak X is allowed to be a general topological space satisfying only a mild condition ("quasi-Polish"), and the output space can be either another quasi-Polish space mathfrak Y or a topological vector space E. Similarly to arXiv:2109.13512v4, we show furthermore that our neural network architectures can be projected down to "finite dimensional" subspaces with any desirable accuracy, thus obtaining approximating networks that are easy to implement and allow for fast computation and fitting. The resulting neural network architecture is therefore applicable for prediction tasks based on functional data. To the best of our knowledge, this is the first result which deals with such a wide class of input/output spaces and simultaneously guarantees the numerical feasibility of the ensuing architectures. Finally, we prove an obstruction result which indicates that the category of quasi-Polish spaces is in a certain sense the correct category to work with if one aims at constructing approximating architectures on infinite-dimensional spaces mathfrak X which, at the same time, have sufficient expressive power to approximate continuous functions on mathfrak X, are specified by a finite number of parameters only and are "stable" with respect to these parameters.