# IB-UQ: Information bottleneck based uncertainty quantification for neural function regression and neural operator learning

arXiv: https://arxiv.org/abs/2302.03271 · HTML: https://mlsanity.com/paper/2302.03271/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 62 references that are in the map.
- First posted on arXiv 7 Feb 2023.

## Builds on (most cited first, 30 of 36)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Deep Ensembles: A Loss Landscape Perspective](https://mlsanity.com/paper/1912.02757/) (2019)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons](https://mlsanity.com/paper/2201.07766/) (2022)
- [Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems](https://mlsanity.com/paper/2003.04919/) (2020)
- [Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)](https://mlsanity.com/paper/2001.04872/) (2020)
- [Learning Operators with Coupled Attention](https://mlsanity.com/paper/2201.01032/) (2022)
- [Nonlinear Information Bottleneck](https://mlsanity.com/paper/1705.02436/) (2017)
- [Failure-informed adaptive sampling for PINNs](https://mlsanity.com/paper/2210.00279/) (2022)
- [Reliable extrapolation of deep neural operators informed by physics or sparse observations](https://mlsanity.com/paper/2212.06347/) (2022)
- [Uncertainty in the Variational Information Bottleneck](https://mlsanity.com/paper/1807.00906/) (2018)
- [Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priors](https://mlsanity.com/paper/2203.03048/) (2022)
- [Normalizing field flows: Solving forward and inverse stochastic differential equations using physics-informed flow models](https://mlsanity.com/paper/2108.12956/) (2021)
- [NeuralUQ: A comprehensive library for uncertainty quantification in neural differential equations and operators](https://mlsanity.com/paper/2208.11866/) (2022)

## Built on by (most cited first, 18 of 18)

- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [Conformalized-DeepONet: A Distribution-Free Framework for Uncertainty Quantification in Deep Operator Networks](https://mlsanity.com/paper/2402.15406/) (2024)
- [Probabilistic neural operators for functional uncertainty quantification](https://mlsanity.com/paper/2502.12902/) (2025)
- [Operator Learning: A Statistical Perspective](https://mlsanity.com/paper/2504.03503/) (2025)
- [Using Uncertainty Quantification to Characterize and Improve Out-of-Domain Learning for PDEs](https://mlsanity.com/paper/2403.10642/) (2024)
- [Learning in PINNs: Phase transition, diffusion equilibrium, and generalization](https://mlsanity.com/paper/2403.18494/) (2024)
- [A physics-informed transformer neural operator for learning generalized solutions of initial boundary value problems](https://mlsanity.com/paper/2412.09009/) (2024)
- [Uncertainty quantification for deeponets with ensemble kalman inversion](https://mlsanity.com/paper/2403.03444/) (2024)
- [Alpha-VI DeepONet: A prior-robust variational Bayesian approach for enhancing DeepONets with uncertainty quantification](https://mlsanity.com/paper/2408.00681/) (2024)
- [Stochastic Operator Network: A Stochastic Maximum Principle Based Approach to Operator Learning](https://mlsanity.com/paper/2507.10401/) (2025)
- [Split Conformal Prediction in the Function Space with Neural Operators](https://mlsanity.com/paper/2509.04623/) (2025)
- [Orthogonal greedy algorithm for linear operator learning with shallow neural network](https://mlsanity.com/paper/2501.02791/) (2025)
- [Stable Derivative Free Gaussian Mixture Variational Inference for Bayesian Inverse Problems](https://mlsanity.com/paper/2501.04259/) (2025)
- [Self-composing neural operators for high-frequency and multiscale PDE surrogates](https://mlsanity.com/paper/2508.20650/) (2025)
- [LVM-GP: Uncertainty-Aware PDE Solver via coupling latent variable model and Gaussian process](https://mlsanity.com/paper/2507.22493/) (2025)
- [Deep set based operator learning with uncertainty quantification](https://mlsanity.com/paper/2509.25646/) (2025)
- [Diffusion-Based Stochastic Operator Networks for Uncertainty Quantification in Stochastic Partial Differential Equations](https://mlsanity.com/paper/2605.17107/) (2026)
- [Neural Quantum Spectral Operator Learning for Solving Partial Differential Equations](https://mlsanity.com/paper/2605.27408/) (2026)

## Abstract (arXiv)

We propose a novel framework for uncertainty quantification via information bottleneck (IB-UQ) for scientific machine learning tasks, including deep neural network (DNN) regression and neural operator learning (DeepONet). Specifically, we incorporate the bottleneck by a confidence-aware encoder, which encodes inputs into latent representations according to the confidence of the input data belonging to the region where training data is located, and utilize a Gaussian decoder to predict means and variances of outputs conditional on representation variables. Furthermore, we propose a data augmentation based information bottleneck objective which can enhance the quantification quality of the extrapolation uncertainty, and the encoder and decoder can be both trained by minimizing a tractable variational bound of the objective. In comparison to uncertainty quantification (UQ) methods for scientific learning tasks that rely on Bayesian neural networks with Hamiltonian Monte Carlo posterior estimators, the model we propose is computationally efficient, particularly when dealing with large-scale data sets. The effectiveness of the IB-UQ model has been demonstrated through several representative examples, such as regression for discontinuous functions, real-world data set regression, learning nonlinear operators for partial differential equations, and a large-scale climate model. The experimental results indicate that the IB-UQ model can handle noisy data, generate robust predictions, and provide confident uncertainty evaluation for out-of-distribution data.