# Using Uncertainty Quantification to Characterize and Improve Out-of-Domain Learning for PDEs

arXiv: https://arxiv.org/abs/2403.10642 · HTML: https://mlsanity.com/paper/2403.10642/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 66 references that are in the map.
- First posted on arXiv 15 Mar 2024.

## Builds on (most cited first, 30 of 41)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Fourier Neural Operator with Learned Deformations for PDEs on General Geometries](https://mlsanity.com/paper/2207.05209/) (2022)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [Multiwavelet-based Operator Learning for Differential Equations](https://mlsanity.com/paper/2109.13459/) (2021)
- [Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons](https://mlsanity.com/paper/2201.07766/) (2022)
- [Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior](https://mlsanity.com/paper/2306.00258/) (2023)
- [A Mathematical Guide to Operator Learning](https://mlsanity.com/paper/2312.14688/) (2023)
- [Continuous PDE Dynamics Forecasting with Implicit Neural Representations](https://mlsanity.com/paper/2209.14855/) (2022)
- [A Unified Theory of Diversity in Ensemble Learning](https://mlsanity.com/paper/2301.03962/) (2023)
- [Bayesian Uncertainty Estimation for Batch Normalized Deep Networks](https://mlsanity.com/paper/1802.06455/) (2018)
- [Learning Physical Models that Can Respect Conservation Laws](https://mlsanity.com/paper/2302.11002/) (2023)
- [DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation](https://mlsanity.com/paper/2101.05544/) (2021)
- [Bayesian Neural Ordinary Differential Equations](https://mlsanity.com/paper/2012.07244/) (2020)
- [Learning Dissipative Dynamics in Chaotic Systems](https://mlsanity.com/paper/2106.06898/) (2021)
- [AtmoRep: A stochastic model of atmosphere dynamics using large scale representation learning](https://mlsanity.com/paper/2308.13280/) (2023)
- [Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priors](https://mlsanity.com/paper/2203.03048/) (2022)
- [Learning differentiable solvers for systems with hard constraints](https://mlsanity.com/paper/2207.08675/) (2022)
- [Out-of-distributional risk bounds for neural operators with applications to the Helmholtz equation](https://mlsanity.com/paper/2301.11509/) (2023)
- [Conformal Prediction Bands for Multivariate Functional Data](https://mlsanity.com/paper/2106.01792/) (2021)
- [Reduced order modeling of parametrized systems through autoencoders and SINDy approach: continuation of periodic solutions](https://mlsanity.com/paper/2211.06786/) (2022)
- [INO: Invariant Neural Operators for Learning Complex Physical Systems with Momentum Conservation](https://mlsanity.com/paper/2212.14365/) (2022)

## Built on by (most cited first, 15 of 15)

- [Physics-Constrained Flow Matching: Sampling Generative Models with Hard Constraints](https://mlsanity.com/paper/2506.04171/) (2025)
- [Gradient-Free Generation for Hard-Constrained Systems](https://mlsanity.com/paper/2412.01786/) (2024)
- [Active Learning for Neural PDE Solvers](https://mlsanity.com/paper/2408.01536/) (2024)
- [Zero-Shot Uncertainty Quantification using Diffusion Probabilistic Models](https://mlsanity.com/paper/2408.04718/) (2024)
- [Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generation](https://mlsanity.com/paper/2505.22391/) (2025)
- [End-to-End Probabilistic Framework for Learning with Hard Constraints](https://mlsanity.com/paper/2506.07003/) (2025)
- [Learning PDE Solvers with Physics and Data: A Unifying View of Physics-Informed Neural Networks and Neural Operators](https://mlsanity.com/paper/2601.14517/) (2026)
- [Probabilistic Predictions of Process-Induced Deformation in Carbon/Epoxy Composites Using a Deep Operator Network](https://mlsanity.com/paper/2512.13746/) (2025)
- [Learning Data-Efficient and Generalizable Neural Operators via Fundamental Physics Knowledge](https://mlsanity.com/paper/2602.15184/) (2026)
- [Harnessing AI for Inverse Partial Differential Equation Problems: Past, Present, and Prospects](https://mlsanity.com/paper/2605.16966/) (2026)
- [Geometry-Aware Post-Hoc Uncertainty Quantification in Operator Learning](https://mlsanity.com/paper/2606.17513/) (2026)
- [GeoQ: Geometry-Aware Conditional Quantile Error Estimation for Scientific Surrogate Models](https://mlsanity.com/paper/2608.21652/) (2026)
- [FluidGaussian: Propagating Simulation-Based Uncertainty Toward Functionally-Intelligent 3D Reconstruction](https://mlsanity.com/paper/2603.21356/) (2026)
- [Conformal Uncertainty Quantification Guarantees for Neural Operators](https://mlsanity.com/paper/2608.28515/) (2026)
- [Threshold-Aware Conformal Routing](https://mlsanity.com/paper/2610.02487/) (2026)

## Abstract (arXiv)

Existing work in scientific machine learning (SciML) has shown that data-driven learning of solution operators can provide a fast approximate alternative to classical numerical partial differential equation (PDE) solvers. Of these, Neural Operators (NOs) have emerged as particularly promising. We observe that several uncertainty quantification (UQ) methods for NOs fail for test inputs that are even moderately out-of-domain (OOD), even when the model approximates the solution well for in-domain tasks. To address this limitation, we show that ensembling several NOs can identify high-error regions and provide good uncertainty estimates that are well-correlated with prediction errors. Based on this, we propose a cost-effective alternative, DiverseNO, that mimics the properties of the ensemble by encouraging diverse predictions from its multiple heads in the last feed-forward layer. We then introduce Operator-ProbConserv, a method that uses these well-calibrated UQ estimates within the ProbConserv framework to update the model. Our empirical results show that Operator-ProbConserv enhances OOD model performance for a variety of challenging PDE problems and satisfies physical constraints such as conservation laws.