# Latent Bottlenecked Attentive Neural Processes

arXiv: https://arxiv.org/abs/2211.08458 · HTML: https://mlsanity.com/paper/2211.08458/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 24 references that are in the map.
- First posted on arXiv 15 Nov 2022.

## Builds on (most cited first, 20 of 20)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Perceiver IO: A General Architecture for Structured Inputs & Outputs](https://mlsanity.com/paper/2107.14795/) (2021)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Visual Transformers: Token-based Image Representation and Processing for Computer Vision](https://mlsanity.com/paper/2006.03677/) (2020)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling](https://mlsanity.com/paper/1802.09127/) (2018)
- [Convolutional Conditional Neural Processes](https://mlsanity.com/paper/1910.13556/) (2019)
- [Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling](https://mlsanity.com/paper/2207.04179/) (2022)
- [Coordination Among Neural Modules Through a Shared Global Workspace](https://mlsanity.com/paper/2103.01197/) (2021)
- [Sequential Neural Processes](https://mlsanity.com/paper/1906.10264/) (2019)
- [Bootstrapping Neural Processes](https://mlsanity.com/paper/2008.02956/) (2020)
- [The Gaussian Neural Process](https://mlsanity.com/paper/2101.03606/) (2021)
- [Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processes](https://mlsanity.com/paper/2011.12916/) (2020)
- [The Neural Process Family: Survey, Applications and Perspectives](https://mlsanity.com/paper/2209.00517/) (2022)
- [Meta-Learning surrogate models for sequential decision making](https://mlsanity.com/paper/1903.11907/) (2019)
- [On Contrastive Representations of Stochastic Processes](https://mlsanity.com/paper/2106.10052/) (2021)
- [Recurrent Neural Processes](https://mlsanity.com/paper/1906.05915/) (2019)

## Built on by (most cited first, 26 of 26)

- [AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformers](https://mlsanity.com/paper/2502.09692/) (2025)
- [TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks](https://mlsanity.com/paper/2402.11137/) (2024)
- [Amortized Probabilistic Conditioning for Optimization, Simulation and Inference](https://mlsanity.com/paper/2410.15320/) (2024)
- [Translation Equivariant Transformer Neural Processes](https://mlsanity.com/paper/2406.12409/) (2024)
- [Estimating Interventional Distributions with Uncertain Causal Graphs through Meta-Learning](https://mlsanity.com/paper/2507.05526/) (2025)
- [Beyond Unimodal: Generalising Neural Processes for Multimodal Uncertainty Estimation](https://mlsanity.com/paper/2304.01518/) (2023)
- [Memory Efficient Neural Processes via Constant Memory Attention Block](https://mlsanity.com/paper/2305.14567/) (2023)
- [Approximately Equivariant Neural Processes](https://mlsanity.com/paper/2406.13488/) (2024)
- [ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisition](https://mlsanity.com/paper/2506.07259/) (2025)
- [Dimension Agnostic Neural Processes](https://mlsanity.com/paper/2502.20661/) (2025)
- [Spectral Convolutional Conditional Neural Processes](https://mlsanity.com/paper/2404.13182/) (2024)
- [In-Context In-Context Learning with Transformer Neural Processes](https://mlsanity.com/paper/2406.13493/) (2024)
- [Gridded Transformer Neural Processes for Large Unstructured Spatio-Temporal Data](https://mlsanity.com/paper/2410.06731/) (2024)
- [Efficient Autoregressive Inference for Transformer Probabilistic Models](https://mlsanity.com/paper/2510.09477/) (2025)
- [Adaptive Conditional Quantile Neural Processes](https://mlsanity.com/paper/2305.18777/) (2023)
- [System-Aware Neural ODE Processes for Few-Shot Bayesian Optimization](https://mlsanity.com/paper/2406.02352/) (2024)
- [Tree Cross Attention](https://mlsanity.com/paper/2309.17388/) (2023)
- [Parallelized Spatiotemporal Binding](https://mlsanity.com/paper/2402.17077/) (2024)
- [Exploring Pseudo-Token Approaches in Transformer Neural Processes](https://mlsanity.com/paper/2504.14416/) (2025)
- [Incremental Transformer Neural Processes](https://mlsanity.com/paper/2602.18955/) (2026)
- [Constant Memory Attention Block](https://mlsanity.com/paper/2306.12599/) (2023)
- [Scalable Spatiotemporal Inference with Biased Scan Attention Transformer Neural Processes](https://mlsanity.com/paper/2506.09163/) (2025)
- [Exploring Probabilistic Models for Semi-supervised Learning](https://mlsanity.com/paper/2404.04199/) (2024)
- [Amortising Inference and Meta-Learning Priors in Neural Networks](https://mlsanity.com/paper/2602.08782/) (2026)
- [PRIM: Meta-Learned Bayesian Root Cause Analysis](https://mlsanity.com/paper/2605.08786/) (2026)
- [Revisiting Neural Processes via Fourier Transform and Volterra Series](https://mlsanity.com/paper/2606.01172/) (2026)

## Abstract (arXiv)

Neural Processes (NPs) are popular methods in meta-learning that can estimate predictive uncertainty on target datapoints by conditioning on a context dataset. Previous state-of-the-art method Transformer Neural Processes (TNPs) achieve strong performance but require quadratic computation with respect to the number of context datapoints, significantly limiting its scalability. Conversely, existing sub-quadratic NP variants perform significantly worse than that of TNPs. Tackling this issue, we propose Latent Bottlenecked Attentive Neural Processes (LBANPs), a new computationally efficient sub-quadratic NP variant, that has a querying computational complexity independent of the number of context datapoints. The model encodes the context dataset into a constant number of latent vectors on which self-attention is performed. When making predictions, the model retrieves higher-order information from the context dataset via multiple cross-attention mechanisms on the latent vectors. We empirically show that LBANPs achieve results competitive with the state-of-the-art on meta-regression, image completion, and contextual multi-armed bandits. We demonstrate that LBANPs can trade-off the computational cost and performance according to the number of latent vectors. Finally, we show LBANPs can scale beyond existing attention-based NP variants to larger dataset settings.