# An Approximation Theory for Metric Space-Valued Functions With A View Towards Deep Learning

arXiv: https://arxiv.org/abs/2304.12231 · HTML: https://mlsanity.com/paper/2304.12231/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2023.

## Built on by (most cited first, 9 of 9)

- [Transformers are Universal In-context Learners](https://mlsanity.com/paper/2408.01367/) (2024)
- [Neural Operators Can Play Dynamic Stackelberg Games](https://mlsanity.com/paper/2411.09644/) (2024)
- [Transformers Can Solve Non-Linear and Non-Markovian Filtering Problems in Continuous Time For Conditionally Gaussian Signals](https://mlsanity.com/paper/2310.19603/) (2023)
- [New universal operator approximation theorem for encoder-decoder architectures](https://mlsanity.com/paper/2503.24092/) (2025)
- [On shallow feedforward neural networks with inputs from a topological space](https://mlsanity.com/paper/2504.02321/) (2025)
- [Weighted universal approximation of differentiable maps on infinite-dimensional manifolds](https://mlsanity.com/paper/2606.09820/) (2026)
- [Neural operators approximate strongly continuous convex monotone semigroups](https://mlsanity.com/paper/2609.02727/) (2026)
- [A Closed-Form Formula for Consistent Lipschitz Regression on Metric Spaces with Sparse Neural Network Realizations](https://mlsanity.com/paper/2609.03129/) (2026)
- [From Distributions to Stochastic Processes: Neural Approximation of Measure-Valued Maps](https://mlsanity.com/paper/2609.33649/) (2026)

## Abstract (arXiv)

Motivated by the developing mathematics of deep learning, we build universal functions approximators of continuous maps between arbitrary Polish metric spaces X and Y using elementary functions between Euclidean spaces as building blocks. Earlier results assume that the target space Y is a topological vector space. We overcome this limitation by ``randomization'': our approximators output discrete probability measures over Y. When X and Y are Polish without additional structure, we prove very general qualitative guarantees; when they have suitable combinatorial structure, we prove quantitative guarantees for H"older-like maps, including maps between finite graphs, solution operators to rough differential equations between certain Carnot groups, and continuous non-linear operators between Banach spaces arising in inverse problems. In particular, we show that the required number of Dirac measures is determined by the combinatorial structure of X and Y. For barycentric Y, including Banach spaces, ℝ-trees, Hadamard manifolds, or Wasserstein spaces on Polish metric spaces, our approximators reduce to Y-valued functions. When the Euclidean approximators are neural networks, our constructions generalize transformer networks, providing a new probabilistic viewpoint of geometric deep learning.