# Learning High-Dimensional Parametric Maps via Reduced Basis Adaptive Residual Networks

arXiv: https://arxiv.org/abs/2112.07096 · HTML: https://mlsanity.com/paper/2112.07096/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 59 references that are in the map.
- First posted on arXiv 14 Dec 2021.

## Builds on (most cited first, 20 of 20)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [Deep Neural Networks Motivated by Partial Differential Equations](https://mlsanity.com/paper/1804.04272/) (2018)
- [The Random Feature Model for Input-Output Maps between Banach Spaces](https://mlsanity.com/paper/2005.10224/) (2020)
- [ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction](https://mlsanity.com/paper/2105.10446/) (2021)
- [ResNet with one-neuron hidden layers is a Universal Approximator](https://mlsanity.com/paper/1806.10909/) (2018)
- [POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition](https://mlsanity.com/paper/2101.11845/) (2021)
- [Deep Learning via Dynamical Systems: An Approximation Perspective](https://mlsanity.com/paper/1912.10382/) (2019)
- [Convergence Rates for Learning Linear Operators from Noisy Data](https://mlsanity.com/paper/2108.12515/) (2021)
- [Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs](https://mlsanity.com/paper/2011.15110/) (2020)
- [Shallow-to-Deep Training for Neural Machine Translation](https://mlsanity.com/paper/2010.03737/) (2020)
- [A computational framework for infinite-dimensional Bayesian inverse problems. Part I: The linearized case, with application to global seismic inversion](https://mlsanity.com/paper/1308.1313/) (2013)
- [hIPPYlib: An Extensible Software Framework for Large-Scale Inverse Problems Governed by PDEs; Part I: Deterministic Inversion and Linearized Bayesian Inference](https://mlsanity.com/paper/1909.03948/) (2019)
- [Low Rank Saddle Free Newton: A Scalable Method for Stochastic Nonconvex Optimization](https://mlsanity.com/paper/2002.02881/) (2020)
- [Reduced Order Modeling using Shallow ReLU Networks with Grassmann Layers](https://mlsanity.com/paper/2012.09940/) (2020)

## Built on by (most cited first, 16 of 16)

- [Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning](https://mlsanity.com/paper/2206.10745/) (2022)
- [An operator learning perspective on parameter-to-observable maps](https://mlsanity.com/paper/2402.06031/) (2024)
- [Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems](https://mlsanity.com/paper/2210.03008/) (2022)
- [Efficient PDE-Constrained optimization under high-dimensional uncertainty using derivative-informed neural operators](https://mlsanity.com/paper/2305.20053/) (2023)
- [Residual-Based Error Corrector Operator to Enhance Accuracy and Reliability of Neural Operator Surrogates of Nonlinear Variational Boundary-Value Problems](https://mlsanity.com/paper/2306.12047/) (2023)
- [Physics-Informed Latent Neural Operator for Real-time Predictions of time-dependent parametric PDEs](https://mlsanity.com/paper/2501.08428/) (2025)
- [Performance of Neural and Polynomial Operator Surrogates](https://mlsanity.com/paper/2604.00689/) (2026)
- [A Framework for Strategic Discovery of Credible Neural Network Surrogate Models under Uncertainty](https://mlsanity.com/paper/2403.08901/) (2024)
- [Active Learning of Model Discrepancy with Bayesian Experimental Design](https://mlsanity.com/paper/2502.05372/) (2025)
- [PTPI-DL-ROMs: pre-trained physics-informed deep learning-based reduced order models for nonlinear parametrized PDEs](https://mlsanity.com/paper/2405.08558/) (2024)
- [Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order models](https://mlsanity.com/paper/2409.05709/) (2024)
- [Bayesian Experimental Design for Model Discrepancy Calibration: An Auto-Differentiable Ensemble Kalman Inversion Approach](https://mlsanity.com/paper/2504.20319/) (2025)
- [Reduced-Basis Deep Operator Learning for Parametric PDEs with Independently Varying Boundary and Source Data](https://mlsanity.com/paper/2511.18260/) (2025)
- [Sequential Bayesian Optimal Experimental Design in Infinite Dimensions via Policy Gradient Reinforcement Learning](https://mlsanity.com/paper/2601.05868/) (2026)
- [Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models](https://mlsanity.com/paper/2412.09942/) (2024)
- [Bayesian Experimental Design for Model Discrepancy Calibration: A Rivalry between Kullback--Leibler Divergence and Wasserstein Distance](https://mlsanity.com/paper/2601.16425/) (2026)

## Abstract (arXiv)

We propose a scalable framework for the learning of high-dimensional parametric maps via adaptively constructed residual network (ResNet) maps between reduced bases of the inputs and outputs. When just few training data are available, it is beneficial to have a compact parametrization in order to ameliorate the ill-posedness of the neural network training problem. By linearly restricting high-dimensional maps to informed reduced bases of the inputs, one can compress high-dimensional maps in a constructive way that can be used to detect appropriate basis ranks, equipped with rigorous error estimates. A scalable neural network learning framework is thus to learn the nonlinear compressed reduced basis mapping. Unlike the reduced basis construction, however, neural network constructions are not guaranteed to reduce errors by adding representation power, making it difficult to achieve good practical performance. Inspired by recent approximation theory that connects ResNets to sequential minimizing flows, we present an adaptive ResNet construction algorithm. This algorithm allows for depth-wise enrichment of the neural network approximation, in a manner that can achieve good practical performance by first training a shallow network and then adapting. We prove universal approximation of the associated neural network class for L2_nu functions on compact sets. Our overall framework allows for constructive means to detect appropriate breadth and depth, and related compact parametrizations of neural networks, significantly reducing the need for architectural hyperparameter tuning. Numerical experiments for parametric PDE problems and a 3D CFD wing design optimization parametric map demonstrate that the proposed methodology can achieve remarkably high accuracy for limited training data, and outperformed other neural network strategies we compared against.