mlsanityData to 14 Oct 2026

Latent Bottlenecked Attentive Neural Processes

arXiv 2211.08458 ↗First posted 15 Nov 2022cs.LG cs.AIcited by 26

Not on the map. Why?

Abstract (arXiv)

Neural Processes (NPs) are popular methods in meta-learning that can estimate predictive uncertainty on target datapoints by conditioning on a context dataset. Previous state-of-the-art method Transformer Neural Processes (TNPs) achieve strong performance but require quadratic computation with respect to the number of context datapoints, significantly limiting its scalability. Conversely, existing sub-quadratic NP variants perform significantly worse than that of TNPs. Tackling this issue, we propose Latent Bottlenecked Attentive Neural Processes (LBANPs), a new computationally efficient sub-quadratic NP variant, that has a querying computational complexity independent of the number of context datapoints. The model encodes the context dataset into a constant number of latent vectors on which self-attention is performed. When making predictions, the model retrieves higher-order information from the context dataset via multiple cross-attention mechanisms on the latent vectors. We empirically show that LBANPs achieve results competitive with the state-of-the-art on meta-regression, image completion, and contextual multi-armed bandits. We demonstrate that LBANPs can trade-off the computational cost and performance according to the number of latent vectors. Finally, we show LBANPs can scale beyond existing attention-based NP variants to larger dataset settings.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 20 of its 24 references that are in the map. First posted on arXiv 15 Nov 2022.

Builds on

20 of 24 references.

References · 20

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  3. 2017Deep Setscited by 1,421
  4. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  5. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 629
  6. 2018Conditional Neural Processescited by 426
  7. 2020Visual Transformers: Token-based Image Representation and Processing for Computer Visioncited by 334
  8. 2018Neural Processescited by 311
  9. 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 169
  10. 2019Convolutional Conditional Neural Processescited by 142
  11. 2022Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modelingcited by 127
  12. 2021Coordination Among Neural Modules Through a Shared Global Workspacecited by 85
8 more
  1. 2019Sequential Neural Processescited by 55
  2. 2020Bootstrapping Neural Processescited by 40
  3. 2021The Gaussian Neural Processcited by 34
  4. 2020Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processescited by 29
  5. 2022The Neural Process Family: Survey, Applications and Perspectivescited by 23
  6. 2019Meta-Learning surrogate models for sequential decision makingcited by 14
  7. 2021On Contrastive Representations of Stochastic Processescited by 12
  8. 2019Recurrent Neural Processescited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 10; 2025: 7; 2026: 4.

26 papers.

Sort:

  1. 2025AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformerscited by 53
  2. 2024TuneTables: Context Optimization for Scalable Prior-Data Fitted Networkscited by 47
  3. 2024Amortized Probabilistic Conditioning for Optimization, Simulation and Inferencecited by 23
  4. 2024Translation Equivariant Transformer Neural Processescited by 13
  5. 2025Estimating Interventional Distributions with Uncertain Causal Graphs through Meta-Learningcited by 13
  6. 2023Beyond Unimodal: Generalising Neural Processes for Multimodal Uncertainty Estimationcited by 11
  7. 2023Memory Efficient Neural Processes via Constant Memory Attention Blockcited by 9
  8. 2024Approximately Equivariant Neural Processescited by 8
  9. 2025ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisitioncited by 8
  10. 2025Dimension Agnostic Neural Processescited by 5
  11. 2024Spectral Convolutional Conditional Neural Processescited by 4
  12. 2024In-Context In-Context Learning with Transformer Neural Processescited by 4
14 more
  1. 2024Gridded Transformer Neural Processes for Large Unstructured Spatio-Temporal Datacited by 4
  2. 2025Efficient Autoregressive Inference for Transformer Probabilistic Modelscited by 4
  3. 2023Adaptive Conditional Quantile Neural Processescited by 3
  4. 2024System-Aware Neural ODE Processes for Few-Shot Bayesian Optimizationcited by 3
  5. 2023Tree Cross Attentioncited by 2
  6. 2024Parallelized Spatiotemporal Bindingcited by 2
  7. 2025Exploring Pseudo-Token Approaches in Transformer Neural Processescited by 2
  8. 2026Incremental Transformer Neural Processescited by 2
  9. 2023Constant Memory Attention Blockcited by 1
  10. 2025Scalable Spatiotemporal Inference with Biased Scan Attention Transformer Neural Processescited by 1
  11. 2024Exploring Probabilistic Models for Semi-supervised Learningcited by 0
  12. 2026Amortising Inference and Meta-Learning Priors in Neural Networkscited by 0
  13. 2026PRIM: Meta-Learned Bayesian Root Cause Analysiscited by 0
  14. 2026Revisiting Neural Processes via Fourier Transform and Volterra Seriescited by 0