# Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problem

arXiv: https://arxiv.org/abs/2211.08875 · HTML: https://mlsanity.com/paper/2211.08875/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 55 references that are in the map.
- First posted on arXiv 16 Nov 2022.

## Builds on (most cited first, 12 of 12)

- [A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings](https://mlsanity.com/paper/2002.03689/) (2020)
- [Optimal Rates For Regularization Of Statistical Inverse Learning Problems](https://mlsanity.com/paper/1604.04054/) (2016)
- [Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces](https://mlsanity.com/paper/2205.14027/) (2022)
- [Operator-valued Kernels for Learning from Functional Response Data](https://mlsanity.com/paper/1510.08231/) (2015)
- [Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spaces](https://mlsanity.com/paper/1801.06720/) (2018)
- [Convergence Rates for Learning Linear Operators from Noisy Data](https://mlsanity.com/paper/2108.12515/) (2021)
- [Optimal Rates for Regularized Conditional Mean Embedding Learning](https://mlsanity.com/paper/2208.01711/) (2022)
- [Learning Theory for Distribution Regression](https://mlsanity.com/paper/1411.2066/) (2014)
- [A Rigorous Theory of Conditional Mean Embeddings](https://mlsanity.com/paper/1912.00671/) (2019)
- [Nonparametric approximation of conditional expectation operators](https://mlsanity.com/paper/2012.12917/) (2020)
- [Minimax Optimal Kernel Operator Learning via Multilevel Training](https://mlsanity.com/paper/2209.14430/) (2022)
- [The linear conditional expectation in Hilbert space](https://mlsanity.com/paper/2008.12070/) (2020)

## Built on by (most cited first, 28 of 28)

- [Operator Learning: Algorithms and Analysis](https://mlsanity.com/paper/2402.15715/) (2024)
- [Error Bounds for Learning with Vector-Valued Random Features](https://mlsanity.com/paper/2305.17170/) (2023)
- [Towards Optimal Sobolev Norm Rates for the Vector-Valued Regularized Least-Squares Algorithm](https://mlsanity.com/paper/2312.07186/) (2023)
- [Machine Learning for Partial Differential Equations](https://mlsanity.com/paper/2303.17078/) (2023)
- [Operator Learning Using Random Features: A Tool for Scientific Computing](https://mlsanity.com/paper/2408.06526/) (2024)
- [Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithms](https://mlsanity.com/paper/2405.14778/) (2024)
- [Nonparametric Instrumental Regression via Kernel Methods is Minimax Optimal](https://mlsanity.com/paper/2411.19653/) (2024)
- [Optimal Convergence Rates for Neural Operators](https://mlsanity.com/paper/2412.17518/) (2024)
- [Nonparametric Control Koopman Operators](https://mlsanity.com/paper/2405.07312/) (2024)
- [Inverse Problems with Learned Forward Operators](https://mlsanity.com/paper/2311.12528/) (2023)
- [A randomized algorithm to solve reduced rank operator regression](https://mlsanity.com/paper/2312.17348/) (2023)
- [Domain Generalization by Functional Regression](https://mlsanity.com/paper/2302.04724/) (2023)
- [Online Infinite-Dimensional Regression: Learning Linear Operators](https://mlsanity.com/paper/2309.06548/) (2023)
- [Regularized least squares learning with heavy-tailed noise is minimax optimal](https://mlsanity.com/paper/2505.14214/) (2025)
- [On regularized polynomial functional regression](https://mlsanity.com/paper/2311.03036/) (2023)
- [Learned reconstruction methods for inverse problems: sample error estimates](https://mlsanity.com/paper/2312.14078/) (2023)
- [Kernel-Based Optimal Control: An Infinitesimal Generator Approach](https://mlsanity.com/paper/2412.01591/) (2024)
- [A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learning](https://mlsanity.com/paper/2509.11070/) (2025)
- [Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators](https://mlsanity.com/paper/2603.00971/) (2026)
- [Optimal Multiscale Learning of Linear Operators](https://mlsanity.com/paper/2606.16913/) (2026)
- [Verifiable Regularity Criterion for Conditional Expectation Operators and Conditional Mean Embeddings with Applications to Nonparametric Regression, Bayesian Inverse Problems, and Koopman Operators](https://mlsanity.com/paper/2608.06155/) (2026)
- [Learning convolution operators on compact Abelian groups](https://mlsanity.com/paper/2501.05279/) (2025)
- [Kernel Mean Embedding Topology: Weak and Strong Forms for Stochastic Kernels and Implications for Model Learning](https://mlsanity.com/paper/2502.13486/) (2025)
- [Operator Learning for Schr"odinger Equation: Unitarity, Error Bounds, and Time Generalization](https://mlsanity.com/paper/2505.18288/) (2025)
- [Operator Models for Continuous-Time Offline Reinforcement Learning](https://mlsanity.com/paper/2511.10383/) (2025)
- [Kernel-Based Learning of Safety Barriers](https://mlsanity.com/paper/2601.12002/) (2026)
- [Safety Certification is Classification](https://mlsanity.com/paper/2605.06087/) (2026)
- [Near-Optimal Learning of Gaussian Sobolev Operators](https://mlsanity.com/paper/2607.11921/) (2026)

## Abstract (arXiv)

We consider the problem of learning a linear operator θ between two Hilbert spaces from empirical observations, which we interpret as least squares regression in infinite dimensions. We show that this goal can be reformulated as an inverse problem for θ with the feature that its forward operator is generally non-compact (even if θ is assumed to be compact or of p-Schatten class). However, we prove that, in terms of spectral properties and regularisation theory, this inverse problem is equivalent to the known compact inverse problem associated with scalar response regression. Our framework allows for the elegant derivation of dimension-free rates for generic learning algorithms under H"older-type source conditions. The proofs rely on the combination of techniques from kernel regression with recent results on concentration of measure for sub-exponential Hilbertian random variables. The obtained rates hold for a variety of practically-relevant scenarios in functional regression as well as nonlinear regression with operator-valued kernels and match those of classical kernel regression with scalar response.