# Composite Bayesian Optimization In Function Spaces Using NEON -- Neural Epistemic Operator Networks

arXiv: https://arxiv.org/abs/2404.03099 · HTML: https://mlsanity.com/paper/2404.03099/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 48 references that are in the map.
- First posted on arXiv 3 Apr 2024.

## Builds on (most cited first, 27 of 27)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [MIONet: Learning multiple-input operators via tensor product](https://mlsanity.com/paper/2202.06137/) (2022)
- [Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2006.05078/) (2020)
- [Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons](https://mlsanity.com/paper/2201.07766/) (2022)
- [Unexpected Improvements to Expected Improvement for Bayesian Optimization](https://mlsanity.com/paper/2310.20708/) (2023)
- [Bayesian Neural Networks: An Introduction and Survey](https://mlsanity.com/paper/2006.12024/) (2020)
- [Epistemic Neural Networks](https://mlsanity.com/paper/2107.08924/) (2021)
- [Learning Operators with Coupled Attention](https://mlsanity.com/paper/2201.01032/) (2022)
- [Improved architectures and training algorithms for deep operator networks](https://mlsanity.com/paper/2110.01654/) (2021)
- [NOMAD: Nonlinear Manifold Decoders for Operator Learning](https://mlsanity.com/paper/2206.03551/) (2022)
- [A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks](https://mlsanity.com/paper/1912.10481/) (2019)
- [Bayesian Optimization of Composite Functions](https://mlsanity.com/paper/1906.01537/) (2019)
- [Parallel Bayesian Global Optimization of Expensive Functions](https://mlsanity.com/paper/1602.05149/) (2016)
- [Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies](https://mlsanity.com/paper/2006.06091/) (2020)
- [Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priors](https://mlsanity.com/paper/2203.03048/) (2022)
- [Bayesian Optimization with High-Dimensional Outputs](https://mlsanity.com/paper/2106.12997/) (2021)
- [BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization](https://mlsanity.com/paper/1910.06403/) (2019)
- [Optimizing Coverage and Capacity in Cellular Networks using Machine Learning](https://mlsanity.com/paper/2010.13710/) (2020)
- [Deep Learning for Bayesian Optimization of Scientific Problems with High-Dimensional Structure](https://mlsanity.com/paper/2104.11667/) (2021)
- [Interferobot: aligning an optical interferometer by a reinforcement learning agent](https://mlsanity.com/paper/2006.02252/) (2020)
- [Joint Composite Latent Space Bayesian Optimization](https://mlsanity.com/paper/2311.02213/) (2023)

## Built on by (most cited first, 10 of 10)

- [Deep Learning Alternatives of the Kolmogorov Superposition Theorem](https://mlsanity.com/paper/2410.01990/) (2024)
- [Active operator learning with predictive uncertainty quantification for partial differential equations](https://mlsanity.com/paper/2503.03178/) (2025)
- [SPIEDiff: robust learning of long-time macroscopic dynamics from short-time particle simulations with quantified epistemic uncertainty](https://mlsanity.com/paper/2505.13501/) (2025)
- [Thompson Sampling in Function Spaces via Neural Operators](https://mlsanity.com/paper/2506.21894/) (2025)
- [Multimodal Scientific Learning Beyond Diffusions and Flows](https://mlsanity.com/paper/2602.00960/) (2026)
- [Active Learning Design: Modeling Force Output for Axisymmetric Soft Pneumatic Actuators](https://mlsanity.com/paper/2504.01156/) (2025)
- [Leveraging AI modelling for FDS with Simvue: monitor and optimise for more sustainable simulations](https://mlsanity.com/paper/2509.26139/) (2025)
- [A Mutual Information Lower Bound for Multimodal Regression Active Learning](https://mlsanity.com/paper/2605.14917/) (2026)
- [Structure-preserving uncertainty quantification for GENERIC dynamics](https://mlsanity.com/paper/2608.12624/) (2026)
- [Sequential operator learning under dependent data](https://mlsanity.com/paper/2608.24426/) (2026)

## Abstract (arXiv)

Operator learning is a rising field of scientific computing where inputs or outputs of a machine learning model are functions defined in infinite-dimensional spaces. In this paper, we introduce NEON (Neural Epistemic Operator Networks), an architecture for generating predictions with uncertainty using a single operator network backbone, which presents orders of magnitude less trainable parameters than deep ensembles of comparable performance. We showcase the utility of this method for sequential decision-making by examining the problem of composite Bayesian Optimization (BO), where we aim to optimize a function f=gcirc h, where h:X→ C(Y,ℝds) is an unknown map which outputs elements of a function space, and g: C(Y,ℝds)→ ℝ is a known and cheap-to-compute functional. By comparing our approach to other state-of-the-art methods on toy and real world scenarios, we demonstrate that NEON achieves state-of-the-art performance while requiring orders of magnitude less trainable parameters.