# Bridge the Inference Gaps of Neural Processes via Expectation Maximization

arXiv: https://arxiv.org/abs/2501.03264 · HTML: https://mlsanity.com/paper/2501.03264/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 46 references that are in the map.
- First posted on arXiv 4 Jan 2025.

## Builds on (most cited first, 27 of 27)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [VAE with a VampPrior](https://mlsanity.com/paper/1705.07120/) (2017)
- [Attend, Infer, Repeat: Fast Scene Understanding with Generative Models](https://mlsanity.com/paper/1603.08575/) (2016)
- [Convolutional Conditional Neural Processes](https://mlsanity.com/paper/1910.13556/) (2019)
- [Inference Suboptimality in Variational Autoencoders](https://mlsanity.com/paper/1801.03558/) (2018)
- [Meta-Learning Probabilistic Inference For Prediction](https://mlsanity.com/paper/1805.09921/) (2018)
- [Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processes](https://mlsanity.com/paper/2007.01332/) (2020)
- [Auxiliary Deep Generative Models](https://mlsanity.com/paper/1602.05473/) (2016)
- [Reweighted Wake-Sleep](https://mlsanity.com/paper/1406.2751/) (2014)
- [The Functional Neural Process](https://mlsanity.com/paper/1906.08324/) (2019)
- [Bootstrapping Neural Processes](https://mlsanity.com/paper/2008.02956/) (2020)
- [Doubly Stochastic Variational Inference for Neural Processes with Hierarchical Latent Variables](https://mlsanity.com/paper/2008.09469/) (2020)
- [Factor Analysis, Probabilistic Principal Component Analysis, Variational Inference, and Variational Autoencoder: Tutorial and Survey](https://mlsanity.com/paper/2101.00734/) (2021)
- [Variational Auto-encoded Deep Gaussian Processes](https://mlsanity.com/paper/1511.06455/) (2015)
- [Function Contrastive Learning of Transferable Meta-Representations](https://mlsanity.com/paper/2010.07093/) (2020)
- [Model-based Meta Reinforcement Learning using Graph Structured Surrogate Models](https://mlsanity.com/paper/2102.08291/) (2021)
- [Group Equivariant Conditional Neural Processes](https://mlsanity.com/paper/2102.08759/) (2021)
- [Neural Processes with Stochastic Attention: Paying more attention to the context dataset](https://mlsanity.com/paper/2204.05449/) (2022)
- [Meta-Learning surrogate models for sequential decision making](https://mlsanity.com/paper/1903.11907/) (2019)
- [Nested Variational Inference](https://mlsanity.com/paper/2106.11302/) (2021)
- [Reweighted Expectation Maximization](https://mlsanity.com/paper/1906.05850/) (2019)
- [Uncertainty in Neural Processes](https://mlsanity.com/paper/2010.03753/) (2020)

## Built on by (most cited first, 8 of 8)

- [Model Predictive Task Sampling for Efficient and Robust Adaptation](https://mlsanity.com/paper/2501.11039/) (2025)
- [Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environments](https://mlsanity.com/paper/2504.19139/) (2025)
- [Dimension Agnostic Neural Processes](https://mlsanity.com/paper/2502.20661/) (2025)
- [Distance-informed Neural Processes](https://mlsanity.com/paper/2508.18903/) (2025)
- [Probabilistic Interactive 3D Segmentation with Hierarchical Neural Processes](https://mlsanity.com/paper/2505.01726/) (2025)
- [Prior-Fitted Functional Flow: In-Context Generative Models for Pharmacokinetics](https://mlsanity.com/paper/2604.17670/) (2026)
- [Revisiting Neural Processes via Fourier Transform and Volterra Series](https://mlsanity.com/paper/2606.01172/) (2026)
- [Bridging Risk Approximation Gaps in Model Predictive Task Sampling via In-Context Modeling](https://mlsanity.com/paper/2610.14913/) (2026)

## Abstract (arXiv)

The neural process (NP) is a family of computationally efficient models for learning distributions over functions. However, it suffers from under-fitting and shows suboptimal performance in practice. Researchers have primarily focused on incorporating diverse structural inductive biases, textite.g. attention or convolution, in modeling. The topic of inference suboptimality and an analysis of the NP from the optimization objective perspective has hardly been studied in earlier work. To fix this issue, we propose a surrogate objective of the target log-likelihood of the meta dataset within the expectation maximization framework. The resulting model, referred to as the Self-normalized Importance weighted Neural Process (SI-NP), can learn a more accurate functional prior and has an improvement guarantee concerning the target log-likelihood. Experimental results show the competitive performance of SI-NP over other NPs objectives and illustrate that structural inductive biases, such as attention modules, can also augment our method to achieve SOTA performance. Our code is available at urlhttps://github.com/hhq123gogogo/SINPs.