# Operator Learning of Lipschitz Operators: An Information-Theoretic Perspective

arXiv: https://arxiv.org/abs/2406.18794 · HTML: https://mlsanity.com/paper/2406.18794/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 49 references that are in the map.
- First posted on arXiv 26 Jun 2024.

## Builds on (most cited first, 23 of 23)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [MIONet: Learning multiple-input operators via tensor product](https://mlsanity.com/paper/2202.06137/) (2022)
- [Optimal approximation of piecewise smooth functions using deep ReLU neural networks](https://mlsanity.com/paper/1709.05289/) (2017)
- [Deep Neural Network Approximation Theory](https://mlsanity.com/paper/1901.02220/) (2019)
- [Optimal Approximation with Sparsely Connected Deep Neural Networks](https://mlsanity.com/paper/1705.01714/) (2017)
- [Operator Learning: Algorithms and Analysis](https://mlsanity.com/paper/2402.15715/) (2024)
- [A Theoretical Analysis of Deep Neural Networks and Parametric PDEs](https://mlsanity.com/paper/1904.00377/) (2019)
- [Deep Nonparametric Estimation of Operators between Infinite Dimensional Spaces](https://mlsanity.com/paper/2201.00217/) (2022)
- [Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations](https://mlsanity.com/paper/2112.08125/) (2021)
- [Operator learning with PCA-Net: upper and lower complexity bounds](https://mlsanity.com/paper/2303.16317/) (2023)
- [Out-of-distributional risk bounds for neural operators with applications to the Helmholtz equation](https://mlsanity.com/paper/2301.11509/) (2023)
- [Two-layer neural networks with values in a Banach space](https://mlsanity.com/paper/2105.02095/) (2021)
- [Deep Operator Network Approximation Rates for Lipschitz Operators](https://mlsanity.com/paper/2307.09835/) (2023)
- [Error Bounds for Learning with Vector-Valued Random Features](https://mlsanity.com/paper/2305.17170/) (2023)
- [Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysis](https://mlsanity.com/paper/2210.13300/) (2022)
- [An operator learning perspective on parameter-to-observable maps](https://mlsanity.com/paper/2402.06031/) (2024)
- [Variationally Mimetic Operator Networks](https://mlsanity.com/paper/2209.12871/) (2022)
- [Mixtures of Neural Operators Reduce Active Complexity in Operator Learning](https://mlsanity.com/paper/2404.09101/) (2024)
- [A general approximation lower bound in Lp norm, with applications to feed-forward neural networks](https://mlsanity.com/paper/2206.04360/) (2022)
- [On efficient algorithms for computing near-best polynomial approximations to high-dimensional, Hilbert-valued functions from limited samples](https://mlsanity.com/paper/2203.13908/) (2022)
- [Solving parametric partial differential equations with deep rectified quadratic unit neural networks](https://mlsanity.com/paper/2203.06973/) (2022)

## Built on by (most cited first, 11 of 11)

- [The Sample Complexity of Learning Lipschitz Operators with respect to Gaussian Measures](https://mlsanity.com/paper/2410.23440/) (2024)
- [Generative Neural Operators of Log-Complexity Can Simultaneously Solve Infinitely Many Convex Programs](https://mlsanity.com/paper/2508.14995/) (2025)
- [KANO: Kolmogorov-Arnold Neural Operator](https://mlsanity.com/paper/2509.16825/) (2025)
- [Towards Sharp Minimax Risk Bounds for Operator Learning](https://mlsanity.com/paper/2512.17805/) (2025)
- [Theory-to-Practice Gap for Neural Networks and Neural Operators](https://mlsanity.com/paper/2503.18219/) (2025)
- [Polynomial Scaling is Possible For Neural Operator Approximations of Structured Families of BSDEs](https://mlsanity.com/paper/2410.14788/) (2024)
- [Upper Approximation Bounds for Neural Oscillators](https://mlsanity.com/paper/2512.01015/) (2025)
- [Optimal Multiscale Learning of Linear Operators](https://mlsanity.com/paper/2606.16913/) (2026)
- [Upper Generalization Bounds for Neural Oscillators](https://mlsanity.com/paper/2603.09742/) (2026)
- [Near-Optimal Learning of Gaussian Sobolev Operators](https://mlsanity.com/paper/2607.11921/) (2026)
- [Transmission Factors for Lossy Compression of PDE Training Data: Measuring What Reaches a Trained Operator](https://mlsanity.com/paper/2610.06095/) (2026)

## Abstract (arXiv)

Operator learning based on neural operators has emerged as a promising paradigm for the data-driven approximation of operators, mapping between infinite-dimensional Banach spaces. Despite significant empirical progress, our theoretical understanding regarding the efficiency of these approximations remains incomplete. This work addresses the parametric complexity of neural operator approximations for the general class of Lipschitz continuous operators. Motivated by recent findings on the limitations of specific architectures, termed curse of parametric complexity, we here adopt an information-theoretic perspective. Our main contribution establishes lower bounds on the metric entropy of Lipschitz operators in two approximation settings; uniform approximation over a compact set of input functions, and approximation in expectation, with input functions drawn from a probability measure. It is shown that these entropy bounds imply that, regardless of the activation function used, neural operator architectures attaining an approximation accuracy ε must have a size that is exponentially large in ε-1. The size of architectures is here measured by counting the number of encoded bits necessary to store the given model in computational memory. The results of this work elucidate fundamental trade-offs and limitations in operator learning.