mlsanityData to 7 Oct 2026

Using Uncertainty Quantification to Characterize and Improve Out-of-Domain Learning for PDEs

arXiv 2403.10642 ↗First posted 15 Mar 2024cs.LG cs.NA math.NAcited by 14

Not on the map. Why?

Abstract (arXiv)

Existing work in scientific machine learning (SciML) has shown that data-driven learning of solution operators can provide a fast approximate alternative to classical numerical partial differential equation (PDE) solvers. Of these, Neural Operators (NOs) have emerged as particularly promising. We observe that several uncertainty quantification (UQ) methods for NOs fail for test inputs that are even moderately out-of-domain (OOD), even when the model approximates the solution well for in-domain tasks. To address this limitation, we show that ensembling several NOs can identify high-error regions and provide good uncertainty estimates that are well-correlated with prediction errors. Based on this, we propose a cost-effective alternative, DiverseNO, that mimics the properties of the ensemble by encouraging diverse predictions from its multiple heads in the last feed-forward layer. We then introduce Operator-ProbConserv, a method that uses these well-calibrated UQ estimates within the ProbConserv framework to update the model. Our empirical results show that Operator-ProbConserv enhances OOD model performance for a variety of challenging PDE problems and satisfies physical constraints such as conservation laws.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 41 of its 66 references that are in the map. First posted on arXiv 15 Mar 2024.

Builds on

41 of 66 references.

References · 41

  1. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  3. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  4. 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 748
  5. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 722
  6. 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
  7. 2021Physics-Informed Neural Operator for Learning Partial Differential Equationscited by 516
  8. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  9. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 417
  10. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 416
  11. 2021Neural Operator: Learning Maps Between Function Spacescited by 413
  12. 2019Attentive Neural Processescited by 280
29 more
  1. 2021Multiwavelet-based Operator Learning for Differential Equationscited by 229
  2. 2022Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisonscited by 169
  3. 2023Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behaviorcited by 128
  4. 2023A Mathematical Guide to Operator Learningcited by 74
  5. 2022Continuous PDE Dynamics Forecasting with Implicit Neural Representationscited by 71
  6. 2023A Unified Theory of Diversity in Ensemble Learningcited by 71
  7. 2018Bayesian Uncertainty Estimation for Batch Normalized Deep Networkscited by 70
  8. 2023Learning Physical Models that Can Respect Conservation Lawscited by 58
  9. 2021DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimationcited by 48
  10. 2020Bayesian Neural Ordinary Differential Equationscited by 44
  11. 2021Learning Dissipative Dynamics in Chaotic Systemscited by 41
  12. 2022Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priorscited by 39
  13. 2023AtmoRep: A stochastic model of atmosphere dynamics using large scale representation learningcited by 35
  14. 2022Learning differentiable solvers for systems with hard constraintscited by 34
  15. 2023Out-of-distributional risk bounds for neural operators with applications to the Helmholtz equationcited by 32
  16. 2022Reduced order modeling of parametrized systems through autoencoders and SINDy approach: continuation of periodic solutionscited by 29
  17. 2021Conformal Prediction Bands for Multivariate Functional Datacited by 28
  18. 2022INO: Invariant Neural Operators for Learning Complex Physical Systems with Momentum Conservationcited by 28
  19. 2023Uncertainty quantification for noisy inputs-outputs in physics-informed neural networks and neural operatorscited by 28
  20. 2020Diversity inducing Information Bottleneck in Model Ensemblescited by 27
  21. 2024Calibrated Uncertainty Quantification for Operator Learning via Conformal Predictioncited by 25
  22. 2022Approximate Bayesian Neural Operators: Uncertainty Quantification for Parametric PDEscited by 24
  23. 2022Guiding continuous operator learning through Physics-based boundary constraintscited by 21
  24. 2023When are ensembles really effective?cited by 21
  25. 2023IB-UQ: Information bottleneck based uncertainty quantification for neural function regression and neural operator learningcited by 18
  26. 2021How to Avoid Trivial Solutions in Physics-Informed Neural Networkscited by 14
  27. 2023The Interpolating Information Criterion for Overparameterized Modelscited by 11
  28. 2022Monotonicity and Double Descent in Uncertainty Estimation with Gaussian Processescited by 10
  29. 2019n-MeRCI: A new Metric to Evaluate the Correlation Between Predictive Uncertainty and True Errorcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 7.

14 papers.

Sort:

By area:

  1. 2025Physics-Constrained Flow Matching: Sampling Generative Models with Hard Constraintscited by 46
  2. 2024Gradient-Free Generation for Hard-Constrained Systemscited by 35
  3. 2024Active Learning for Neural PDE Solverscited by 22
  4. 2024Zero-Shot Uncertainty Quantification using Diffusion Probabilistic Modelscited by 14
  5. 2025Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generationcited by 11
  6. 2025End-to-End Probabilistic Framework for Learning with Hard Constraintscited by 8
  7. 2026Learning PDE Solvers with Physics and Data: A Unifying View of Physics-Informed Neural Networks and Neural Operatorscited by 6
  8. 2025Probabilistic Predictions of Process-Induced Deformation in Carbon/Epoxy Composites Using a Deep Operator Networkcited by 2
  9. 2026Learning Data-Efficient and Generalizable Neural Operators via Fundamental Physics Knowledgecited by 2
  10. 2026Geometry-Aware Post-Hoc Uncertainty Quantification in Operator Learningcited by 1
  11. 2026GeoQ: Geometry-Aware Conditional Quantile Error Estimation for Scientific Surrogate Modelscited by 1
  12. 2026FluidGaussian: Propagating Simulation-Based Uncertainty Toward Functionally-Intelligent 3D Reconstructioncited by 0
2 more
  1. 2026Harnessing AI for Inverse Partial Differential Equation Problems: Past, Present, and Prospectscited by 0
  2. 2026Conformal Uncertainty Quantification Guarantees for Neural Operatorscited by 0