mlsanityData to 7 Oct 2026

Linearization Turns Neural Operators into Function-Valued Gaussian Processes

arXiv 2406.05072 ↗First posted 7 Jun 2024cs.LG stat.MLcited by 11

Not on the map. Why?

Abstract (arXiv)

Neural operators generalize neural networks to learn mappings between function spaces from data. They are commonly used to learn solution operators of parametric partial differential equations (PDEs) or propagators of time-dependent PDEs. However, to make them useful in high-stakes simulation scenarios, their inherent predictive error must be quantified reliably. We introduce LUNO, a novel framework for approximate Bayesian uncertainty quantification in trained neural operators. Our approach leverages model linearization to push (Gaussian) weight-space uncertainty forward to the neural operator's predictions. We show that this can be interpreted as a probabilistic version of the concept of currying from functional programming, yielding a function-valued (Gaussian) random process belief. Our framework provides a practical yet theoretically sound way to apply existing Bayesian deep learning methods such as the linearized Laplace approximation to neural operators. Just as the underlying neural operator, our approach is resolution-agnostic by design. The method adds minimal prediction overhead, can be applied post-hoc without retraining the network, and scales to large models and datasets. We evaluate these aspects in a case study on Fourier neural operators.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 48 of its 80 references that are in the map. First posted on arXiv 7 Jun 2024.

Builds on

48 of 80 references.

References · 48

  1. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  3. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  4. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  5. 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 748
  6. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 722
  7. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  8. 2021Physics-Informed Neural Operator for Learning Partial Differential Equationscited by 516
  9. 2014New insights and perspectives on the natural gradient methodcited by 435
  10. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 416
  11. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  12. 2020Multipole Graph Neural Operator for Parametric Partial Differential Equationscited by 343
36 more
  1. 2023Geometry-Informed Neural Operator for Large-Scale 3D PDEscited by 293
  2. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  3. 2021Multiwavelet-based Operator Learning for Differential Equationscited by 229
  4. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
  5. 2023Spherical Fourier Neural Operators: Learning Stable Dynamics on the Spherecited by 225
  6. 2023Neural Operators for Accelerating Scientific Simulations and Designcited by 218
  7. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  8. 2018Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adamcited by 131
  9. 2021Solving and Learning Nonlinear PDEs with Gaussian Processescited by 128
  10. 2017Noisy Natural Gradient as Variational Inferencecited by 94
  11. 2019Approximate Inference Turns Deep Networks into Gaussian Processescited by 89
  12. 2020Bayesian Deep Learning via Subnetwork Inferencecited by 85
  13. 2019'In-Between' Uncertainty in Bayesian Neural Networkscited by 79
  14. 2022Lie Point Symmetry Data Augmentation for Neural PDE Solverscited by 76
  15. 2023A Mathematical Guide to Operator Learningcited by 74
  16. 2023Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEscited by 66
  17. 2020Pathwise Conditioning of Gaussian Processescited by 63
  18. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
  19. 2015Operator-valued Kernels for Learning from Functional Response Datacited by 59
  20. 2023Kernel Methods are Competitive for Operator Learningcited by 50
  21. 2022Physics-Informed Gaussian Process Regression Generalizes Linear PDE Solverscited by 40
  22. 2022NeuralUQ: A comprehensive library for uncertainty quantification in neural differential equations and operatorscited by 33
  23. 2023Sparse Cholesky Factorization for Solving Nonlinear PDEs via Gaussian Processescited by 31
  24. 2022Accelerated Linearized Laplace Approximation for Bayesian Deep Learningcited by 27
  25. 2022Approximate Bayesian Neural Operators: Uncertainty Quantification for Parametric PDEscited by 24
  26. 2024Discretization Error of Fourier Neural Operatorscited by 20
  27. 2023Variational Linearized Laplace Approximation for Bayesian Deep Learningcited by 17
  28. 2020Do ideas have shape? Idea registration as the continuous limit of artificial neural networkscited by 16
  29. 2021ViViT: Curvature access through the generalized Gauss-Newton's low-rank structurecited by 16
  30. 2022Variational Bayes Deep Operator Network: A data-driven Bayesian solver for parametric differential equationscited by 11
  31. 2023Forecasting subcritical cylinder wakes with Fourier Neural Operatorscited by 11
  32. 2023Sample Path Regularity of Gaussian Processes from the Covariance Kernelcited by 11
  33. 2020Improving predictions of Bayesian neural nets via local linearizationcited by 7
  34. 2023Resolution-Invariant Image Classification based on Fourier Neural Operatorscited by 7
  35. 2024Neural Operator induced Gaussian Process framework for probabilistic solution of parametric partial differential equationscited by 4
  36. 2024Conditioning of Banach Space Valued Gaussian Random Variables: An Approximation Approach Based on Martingalescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 6.

11 papers.