# Neural Process for Black-Box Model Optimization Under Bayesian Framework

arXiv: https://arxiv.org/abs/2104.02487 · HTML: https://mlsanity.com/paper/2104.02487/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 35 references that are in the map.
- First posted on arXiv 3 Apr 2021.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Population Based Training of Neural Networks](https://mlsanity.com/paper/1711.09846/) (2017)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Scalable Bayesian Optimization Using Deep Neural Networks](https://mlsanity.com/paper/1502.05700/) (2015)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Bayesian Optimization in AlphaGo](https://mlsanity.com/paper/1812.06855/) (2018)
- [Bayesian Optimization using Deep Gaussian Processes](https://mlsanity.com/paper/1905.03350/) (2019)

## Built on by (most cited first, 11 of 11)

- [The Neural Process Family: Survey, Applications and Perspectives](https://mlsanity.com/paper/2209.00517/) (2022)
- [XVO: Generalized Visual Odometry via Cross-Modal Self-Training](https://mlsanity.com/paper/2309.16772/) (2023)
- [Rumor Detection with a novel graph neural network approach](https://mlsanity.com/paper/2403.16206/) (2024)
- [Image Captioning in news report scenario](https://mlsanity.com/paper/2403.16209/) (2024)
- [Advanced AI Framework for Enhanced Detection and Assessment of Abdominal Trauma: Integrating 3D Segmentation with 2D CNN and RNN Models](https://mlsanity.com/paper/2407.16165/) (2024)
- [Scalable Bayesian Optimization via Focalized Sparse Gaussian Processes](https://mlsanity.com/paper/2412.20375/) (2024)
- [Attentive Neural Processes and Batch Bayesian Optimization for Scalable Calibration of Physics-Informed Digital Twins](https://mlsanity.com/paper/2106.15502/) (2021)
- [Sample-Efficient Optimisation with Probabilistic Transformer Surrogates](https://mlsanity.com/paper/2205.13902/) (2022)
- [UniLCD: Unified Local-Cloud Decision-Making via Reinforcement Learning](https://mlsanity.com/paper/2409.11403/) (2024)
- [Deterministic Langevin Unconstrained Optimization with Normalizing Flows](https://mlsanity.com/paper/2310.00745/) (2023)
- [Meta-learning for sample-efficient Bayesian optimisation of fed-batch processes](https://mlsanity.com/paper/2605.05382/) (2026)

## Abstract (arXiv)

There are a large number of optimization problems in physical models where the relationships between model parameters and outputs are unknown or hard to track. These models are named as black-box models in general because they can only be viewed in terms of inputs and outputs, without knowledge of the internal workings. Optimizing the black-box model parameters has become increasingly expensive and time consuming as they have become more complex. Hence, developing effective and efficient black-box model optimization algorithms has become an important task. One powerful algorithm to solve such problem is Bayesian optimization, which can effectively estimates the model parameters that lead to the best performance, and Gaussian Process (GP) has been one of the most widely used surrogate model in Bayesian optimization. However, the time complexity of GP scales cubically with respect to the number of observed model outputs, and GP does not scale well with large parameter dimension either. Consequently, it has been challenging for GP to optimize black-box models that need to query many observations and/or have many parameters. To overcome the drawbacks of GP, in this study, we propose a general Bayesian optimization algorithm that employs a Neural Process (NP) as the surrogate model to perform black-box model optimization, namely, Neural Process for Bayesian Optimization (NPBO). In order to validate the benefits of NPBO, we compare NPBO with four benchmark approaches on a power system parameter optimization problem and a series of seven benchmark Bayesian optimization problems. The results show that the proposed NPBO performs better than the other four benchmark approaches on the power system parameter optimization problem and competitively on the seven benchmark problems.