# Deep learning architectures for nonlinear operator functions and nonlinear inverse problems

arXiv: https://arxiv.org/abs/1912.11090 · HTML: https://mlsanity.com/paper/1912.11090/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Dec 2019.

## Built on by (most cited first, 17 of 17)

- [DiffusionPDE: Generative PDE-Solving Under Partial Observation](https://mlsanity.com/paper/2406.17763/) (2024)
- [Neural Inverse Operators for Solving PDE Inverse Problems](https://mlsanity.com/paper/2301.11167/) (2023)
- [Convergence Rates for Learning Linear Operators from Noisy Data](https://mlsanity.com/paper/2108.12515/) (2021)
- [Out-of-distributional risk bounds for neural operators with applications to the Helmholtz equation](https://mlsanity.com/paper/2301.11509/) (2023)
- [Learning the optimal Tikhonov regularizer for inverse problems](https://mlsanity.com/paper/2106.06513/) (2021)
- [Transformer Meets Boundary Value Inverse Problems](https://mlsanity.com/paper/2209.14977/) (2022)
- [Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysis](https://mlsanity.com/paper/2210.13300/) (2022)
- [Mixtures of Neural Operators Reduce Active Complexity in Operator Learning](https://mlsanity.com/paper/2404.09101/) (2024)
- [Neural Operators Can Play Dynamic Stackelberg Games](https://mlsanity.com/paper/2411.09644/) (2024)
- [One model to solve them all: 2BSDE families via neural operators](https://mlsanity.com/paper/2511.01125/) (2025)
- [Physics-informed sensor coverage through structure preserving machine learning](https://mlsanity.com/paper/2509.10363/) (2025)
- [Simultaneously Solving Infinitely Many LQ Mean Field Games In Hilbert Spaces: The Power of Neural Operators](https://mlsanity.com/paper/2510.20017/) (2025)
- [Learned reconstruction methods for inverse problems: sample error estimates](https://mlsanity.com/paper/2312.14078/) (2023)
- [Multi-robot Multi-source Localization in Complex Flows with Physics-Preserving Environment Models](https://mlsanity.com/paper/2509.14228/) (2025)
- [Rates and architectures for learning geometrically non-trivial operators](https://mlsanity.com/paper/2512.09376/) (2025)
- [The learned range test method for the inverse inclusion problem](https://mlsanity.com/paper/2411.00463/) (2024)
- [NeuralChaos: Optimal Adapted Approximation of Square Integrable Predictable Processes](https://mlsanity.com/paper/2607.14361/) (2026)

## Abstract (arXiv)

We develop a theoretical analysis for special neural network architectures, termed operator recurrent neural networks, for approximating nonlinear functions whose inputs are linear operators. Such functions commonly arise in solution algorithms for inverse boundary value problems. Traditional neural networks treat input data as vectors, and thus they do not effectively capture the multiplicative structure associated with the linear operators that correspond to the data in such inverse problems. We therefore introduce a new family that resembles a standard neural network architecture, but where the input data acts multiplicatively on vectors. Motivated by compact operators appearing in boundary control and the analysis of inverse boundary value problems for the wave equation, we promote structure and sparsity in selected weight matrices in the network. After describing this architecture, we study its representation properties as well as its approximation properties. We furthermore show that an explicit regularization can be introduced that can be derived from the mathematical analysis of the mentioned inverse problems, and which leads to certain guarantees on the generalization properties. We observe that the sparsity of the weight matrices improves the generalization estimates. Lastly, we discuss how operator recurrent networks can be viewed as a deep learning analogue to deterministic algorithms such as boundary control for reconstructing the unknown wavespeed in the acoustic wave equation from boundary measurements.