# Deep Neural Networks Are Effective At Learning High-Dimensional Hilbert-Valued Functions From Limited Data

arXiv: https://arxiv.org/abs/2012.06081 · HTML: https://mlsanity.com/paper/2012.06081/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2020.

## Built on by (most cited first, 22 of 22)

- [Convergence Rates for Learning Linear Operators from Noisy Data](https://mlsanity.com/paper/2108.12515/) (2021)
- [Operator Learning Using Random Features: A Tool for Scientific Computing](https://mlsanity.com/paper/2408.06526/) (2024)
- [Neural Network Training Using ℓ1-Regularization and Bi-fidelity Data](https://mlsanity.com/paper/2105.13011/) (2021)
- [Active Learning for Single Neuron Models with Lipschitz Non-Linearities](https://mlsanity.com/paper/2210.13601/) (2022)
- [A practical existence theorem for reduced order models based on convolutional autoencoders](https://mlsanity.com/paper/2402.00435/) (2024)
- [On efficient algorithms for computing near-best polynomial approximations to high-dimensional, Hilbert-valued functions from limited samples](https://mlsanity.com/paper/2203.13908/) (2022)
- [Learning smooth functions in high dimensions: from sparse polynomials to deep neural networks](https://mlsanity.com/paper/2404.03761/) (2024)
- [CAS4DL: Christoffel Adaptive Sampling for function approximation via Deep Learning](https://mlsanity.com/paper/2208.12190/) (2022)
- [Neural Snowflakes: Universal Latent Graph Inference via Trainable Latent Geometries](https://mlsanity.com/paper/2310.15003/) (2023)
- [A unified framework for learning with nonlinear model classes from arbitrary linear samples](https://mlsanity.com/paper/2311.14886/) (2023)
- [Cauchy Random Features for Operator Learning in Sobolev Space](https://mlsanity.com/paper/2503.00300/) (2025)
- [Response Theory via Generative Score Modeling](https://mlsanity.com/paper/2402.01029/) (2024)
- [A phase transition for finding needles in nonlinear haystacks with LASSO artificial neural networks](https://mlsanity.com/paper/2201.08652/) (2022)
- [Compressive Fourier collocation methods for high-dimensional diffusion equations with periodic boundary conditions](https://mlsanity.com/paper/2206.01255/) (2022)
- [Do stable neural networks exist for classification problems? -- A new view on stability in AI](https://mlsanity.com/paper/2401.07874/) (2024)
- [Physics-informed deep learning and compressive collocation for high-dimensional diffusion-reaction equations: practical existence theory and numerics](https://mlsanity.com/paper/2406.01539/) (2024)
- [Beyond Universal Approximation Theorems: Algorithmic Uniform Approximation by Neural Networks Trained with Noisy Data](https://mlsanity.com/paper/2509.00924/) (2025)
- [Shallow ReLUs Networks in Lp-Type and Sobolev Spaces: Approximation and Path-Norm Controlled Generalization](https://mlsanity.com/paper/2605.18468/) (2026)
- [Surrogate models for diffusion on graphs via sparse polynomials](https://mlsanity.com/paper/2502.06595/) (2025)
- [Rates and architectures for learning geometrically non-trivial operators](https://mlsanity.com/paper/2512.09376/) (2025)
- [Optimal Learning](https://mlsanity.com/paper/2203.15994/) (2022)
- [Reconstruction of frequency-localized functions from pointwise samples via least squares and deep learning](https://mlsanity.com/paper/2502.09794/) (2025)

## Abstract (arXiv)

Accurate approximation of scalar-valued functions from sample points is a key task in computational science. Recently, machine learning with Deep Neural Networks (DNNs) has emerged as a promising tool for scientific computing, with impressive results achieved on problems where the dimension of the data or problem domain is large. This work broadens this perspective, focusing on approximating functions that are Hilbert-valued, i.e. take values in a separable, but typically infinite-dimensional, Hilbert space. This arises in science and engineering problems, in particular those involving solution of parametric Partial Differential Equations (PDEs). Such problems are challenging: 1) pointwise samples are expensive to acquire, 2) the function domain is high dimensional, and 3) the range lies in a Hilbert space. Our contributions are twofold. First, we present a novel result on DNN training for holomorphic functions with so-called hidden anisotropy. This result introduces a DNN training procedure and full theoretical analysis with explicit guarantees on error and sample complexity. The error bound is explicit in three key errors occurring in the approximation procedure: the best approximation, measurement, and physical discretization errors. Our result shows that there exists a procedure (albeit non-standard) for learning Hilbert-valued functions via DNNs that performs as well as, but no better than current best-in-class schemes. It gives a benchmark lower bound for how well DNNs can perform on such problems. Second, we examine whether better performance can be achieved in practice through different types of architectures and training. We provide preliminary numerical results illustrating practical performance of DNNs on parametric PDEs. We consider different parameters, modifying the DNN architecture to achieve better and competitive results, comparing these to current best-in-class schemes.