# Hierarchical Inference and Closure Learning via Adaptive Surrogates for ODEs and PDEs

arXiv: https://arxiv.org/abs/2603.03922 · HTML: https://mlsanity.com/paper/2603.03922/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 59 references that are in the map.
- First posted on arXiv 4 Mar 2026.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [Weak SINDy For Partial Differential Equations](https://mlsanity.com/paper/2007.02848/) (2020)
- [A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applications](https://mlsanity.com/paper/1705.06270/) (2017)
- [Multifidelity deep neural operators for efficient learning of partial differential equations with application to fast inverse design of nanoscale heat transport](https://mlsanity.com/paper/2204.06684/) (2022)
- [A computational framework for infinite-dimensional Bayesian inverse problems. Part I: The linearized case, with application to global seismic inversion](https://mlsanity.com/paper/1308.1313/) (2013)
- [PDE-Transformer: Efficient and Versatile Transformers for Physics Simulations](https://mlsanity.com/paper/2505.24717/) (2025)
- [Data Assimilation: A Mathematical Introduction](https://mlsanity.com/paper/1506.07825/) (2015)
- [A computational framework for infinite-dimensional Bayesian inverse problems: Part II. Stochastic Newton MCMC with application to ice sheet flow inverse problems](https://mlsanity.com/paper/1308.6221/) (2013)
- [Interacting Particle Langevin Algorithm for Maximum Marginal Likelihood Estimation](https://mlsanity.com/paper/2303.13429/) (2023)
- [Sensitivity-Constrained Fourier Neural Operators for Forward and Inverse Problems in Parametric Differential Equations](https://mlsanity.com/paper/2505.08740/) (2025)
- [Deep Learning Closure Models for Large-Eddy Simulation of Flows around Bluff Bodies](https://mlsanity.com/paper/2208.03498/) (2022)
- [Efficient Prior Calibration From Indirect Data](https://mlsanity.com/paper/2405.17955/) (2024)
- [Physics-Informed Deep Inverse Operator Networks for Solving PDE Inverse Problems](https://mlsanity.com/paper/2412.03161/) (2024)
- [Gaussian processes for Bayesian inverse problems associated with linear partial differential equations](https://mlsanity.com/paper/2307.08343/) (2023)
- [BiLO: Bilevel Local Operator Learning for PDE Inverse Problems](https://mlsanity.com/paper/2404.17789/) (2024)
- [Learning-informed parameter identification in nonlinear time-dependent PDEs](https://mlsanity.com/paper/2202.10915/) (2022)
- [A Latent Restoring Force Approach to Nonlinear System Identification](https://mlsanity.com/paper/2109.10681/) (2021)
- [BINDy -- Bayesian identification of nonlinear dynamics with reversible-jump Markov-chain Monte-Carlo](https://mlsanity.com/paper/2408.08062/) (2024)
- [Deep Operator Networks for Bayesian Parameter Estimation in PDEs](https://mlsanity.com/paper/2501.10684/) (2025)
- [A deep surrogate approach to efficient Bayesian inversion in PDE and integral equation models](https://mlsanity.com/paper/1910.01547/) (2019)
- [Efficient Deconvolution in Populational Inverse Problems](https://mlsanity.com/paper/2505.19841/) (2025)
- [A Primer on Variational Inference for Physics-Informed Deep Generative Modelling](https://mlsanity.com/paper/2409.06560/) (2024)
- [Bayesian BiLO: Bilevel Local Operator Learning for Efficient Uncertainty Quantification of Bayesian PDE Inverse Problems with Low-Rank Adaptation](https://mlsanity.com/paper/2507.17019/) (2025)
- [Bayesian grey-box identification of nonlinear convection effects in heat transfer dynamics](https://mlsanity.com/paper/2407.01226/) (2024)

## Abstract (arXiv)

Inverse problems are the task of calibrating models to match data. They play a pivotal role in diverse engineering applications by allowing practitioners to align models with reality. In many applications, engineers and scientists do not have a complete picture of i) the detailed properties of a system (such as material properties, geometry, initial conditions, etc.); ii) the complete laws describing all dynamics at play (such as friction laws, complicated damping phenomena, and general nonlinear interactions). In this paper, we develop a principled methodology for leveraging data from collections of distinct yet related physical systems to jointly estimate the individual model parameters of each system, and learn the shared unknown dynamics in the form of an ML-based closure model. To robustly infer the unknown parameters for each system, we employ a hierarchical Bayesian framework, which allows for the joint inference of multiple systems and their population-level statistics. To learn the closures, we use a maximum marginal likelihood estimate of a neural network embeded within the ODE/PDE formulation of the problem. To realize this framework we utilize the ensemble Metropolis-Adjusted Langevin Algorithm (MALA) for stable and efficient sampling. To mitigate the computational bottleneck of repetitive forward evaluations in solving inverse problems, we introduce a bilevel optimization strategy to simultaneously train a surrogate forward model alongside the inference. Within this framework, we evaluate and compare distinct surrogate architectures, specifically Fourier Neural Operators (FNO) and parametric Physics-Informed Neural Network (PINNs).