# Non-Euclidean Universal Approximation

arXiv: https://arxiv.org/abs/2006.02341 · HTML: https://mlsanity.com/paper/2006.02341/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2020.

## Built on by (most cited first, 29 of 29)

- [Universal Approximation Theorems for Differentiable Geometric Deep Learning](https://mlsanity.com/paper/2101.05390/) (2021)
- [Universal Approximation Under Constraints is Possible with Transformers](https://mlsanity.com/paper/2110.03303/) (2021)
- [Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer](https://mlsanity.com/paper/2201.13094/) (2022)
- [Global universal approximation of functional input maps on weighted spaces](https://mlsanity.com/paper/2306.03303/) (2023)
- [Globally injective and bijective neural operators](https://mlsanity.com/paper/2306.03982/) (2023)
- [Multi-manifold Attention for Vision Transformers](https://mlsanity.com/paper/2207.08569/) (2022)
- [Small Transformers Compute Universal Metric Embeddings](https://mlsanity.com/paper/2209.06788/) (2022)
- [A singular Riemannian geometry approach to Deep Neural Networks I. Theoretical foundations](https://mlsanity.com/paper/2201.09656/) (2021)
- [Chaotic Hedging with Iterated Integrals and Neural Networks](https://mlsanity.com/paper/2209.10166/) (2022)
- [Infinite-dimensional reservoir computing](https://mlsanity.com/paper/2304.00490/) (2023)
- [Do ReLU Networks Have An Edge When Approximating Compactly-Supported Functions?](https://mlsanity.com/paper/2204.11231/) (2022)
- [Universal Regular Conditional Distributions](https://mlsanity.com/paper/2105.07743/) (2021)
- [Capacity Bounds for Hyperbolic Neural Network Representations of Latent Tree Structures](https://mlsanity.com/paper/2308.09250/) (2023)
- [Scalable Message Passing Neural Networks: No Need for Attention in Large Graph Representation Learning](https://mlsanity.com/paper/2411.00835/) (2024)
- [Shape-Informed Clustering of Multi-Dimensional Functional Data via Deep Functional Autoencoders](https://mlsanity.com/paper/2509.22969/) (2025)
- [Learning Free-Surface Flow with Physics-Informed Neural Networks](https://mlsanity.com/paper/2111.09705/) (2021)
- [A Deep Double Ritz Method (D2RM) for solving Partial Differential Equations using Neural Networks](https://mlsanity.com/paper/2211.03627/) (2022)
- [Probing the Effect of Selection Bias on Generalization: A Thought Experiment](https://mlsanity.com/paper/2105.09934/) (2021)
- [Can neural operators always be continuously discretized?](https://mlsanity.com/paper/2412.03393/) (2024)
- [On shallow feedforward neural networks with inputs from a topological space](https://mlsanity.com/paper/2504.02321/) (2025)
- [Neural Multi-Quantile Forecasting for Optimal Inventory Management](https://mlsanity.com/paper/2112.05673/) (2021)
- [Autonomous-Flow-Based Generation](https://mlsanity.com/paper/2511.09902/) (2025)
- [Entropic Riemannian Neural Optimal Transport](https://mlsanity.com/paper/2605.04255/) (2026)
- [Universal Approximation and the Topological Neural Network](https://mlsanity.com/paper/2305.16639/) (2023)
- [Strongly Isomorphic Neural Optimal Transport Across Incomparable Spaces](https://mlsanity.com/paper/2407.14957/) (2024)
- [Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks](https://mlsanity.com/paper/2501.05930/) (2025)
- [Universality of shallow and deep neural networks on non-Euclidean spaces](https://mlsanity.com/paper/2602.23381/) (2026)
- [Neural Operators Can Discover Functional Clusters](https://mlsanity.com/paper/2602.23528/) (2026)
- [Neural operators approximate strongly continuous convex monotone semigroups](https://mlsanity.com/paper/2609.02727/) (2026)

## Abstract (arXiv)

Modifications to a neural network's input and output layers are often required to accommodate the specificities of most practical learning tasks. However, the impact of such changes on architecture's approximation capabilities is largely not understood. We present general conditions describing feature and readout maps that preserve an architecture's ability to approximate any continuous functions uniformly on compacts. As an application, we show that if an architecture is capable of universal approximation, then modifying its final layer to produce binary values creates a new architecture capable of deterministically approximating any classifier. In particular, we obtain guarantees for deep CNNs and deep feed-forward networks. Our results also have consequences within the scope of geometric deep learning. Specifically, when the input and output spaces are Cartan-Hadamard manifolds, we obtain geometrically meaningful feature and readout maps satisfying our criteria. Consequently, commonly used non-Euclidean regression models between spaces of symmetric positive definite matrices are extended to universal DNNs. The same result allows us to show that the hyperbolic feed-forward networks, used for hierarchical learning, are universal. Our result is also used to show that the common practice of randomizing all but the last two layers of a DNN produces a universal family of functions with probability one. We also provide conditions on a DNN's first (resp. last) few layer's connections and activation function which guarantee that these layers can have a width equal to the input (resp. output) space's dimension while not negatively affecting the architecture's approximation capabilities.