mlsanityData to 14 Oct 2026

Attentive Neural Processes

arXiv 1901.05761 ↗First posted 17 Jan 2019cs.LG stat.MLcited by 283

Not on the map. Why?

Abstract (arXiv)

Neural Processes (NPs) (Garnelo et al 2018a;b) approach regression by learning to map a context set of observed input-output pairs to a distribution over regression functions. Each function models the distribution of the output given an input, conditioned on the context. NPs have the benefit of fitting observed data efficiently with linear complexity in the number of context input-output pairs, and can learn a wide family of conditional distributions; they learn predictive distributions conditioned on context sets of arbitrary size. Nonetheless, we show that NPs suffer a fundamental drawback of underfitting, giving inaccurate predictions at the inputs of the observed data they condition on. We address this issue by incorporating attention into NPs, allowing each input location to attend to the relevant context points for the prediction. We show that this greatly improves the accuracy of predictions, results in noticeably faster training, and expands the range of functions that can be modelled.

Key facts. Cited by 283 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 61; 2022: 43; 2023: 52; 2024: 37; 2025: 49; 2026: 40.

283 papers.

Sort:

By area:

  1. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
  2. 2021Transformers Can Do Bayesian Inferencecited by 360
  3. 2022From data to functa: Your data point is a function and you can treat it like onecited by 187
  4. 2022Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modelingcited by 127
  5. 2021Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learningcited by 113
  6. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervisioncited by 112
  7. 2022CARD: Classification and Regression Diffusion Modelscited by 103
  8. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  9. 2021Generative Models as Distributions of Functionscited by 89
  10. 2021Conservative Objective Models for Effective Offline Model-Based Optimizationcited by 88
  11. 2023Score-based Diffusion Models in Function Spacecited by 82
  12. 2021MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Imagescited by 78
38 more
  1. 2023TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023cited by 74
  2. 2022Continuous PDE Dynamics Forecasting with Implicit Neural Representationscited by 73
  3. 2023Learning Physical Models that Can Respect Conservation Lawscited by 59
  4. 2021Neural ODE Processescited by 57
  5. 2022Generalizing to New Physical Systems via Context-Informed Dynamics Modelcited by 50
  6. 2022Neural Diffusion Processescited by 50
  7. 2022Diffusion Generative Models in Infinite Dimensionscited by 49
  8. 2025CausalPFN: Amortized Causal Effect Estimation via In-Context Learningcited by 49
  9. 2022Modeling Temporal Data as Continuous Functions with Stochastic Process Diffusioncited by 39
  10. 2021Offline Model-Based Optimization via Normalized Maximum Likelihood Estimationcited by 38
  11. 2021You Never Cluster Alonecited by 38
  12. 2022Few-Shot Diffusion Modelscited by 38
  13. 2024Gradient-Free Generation for Hard-Constrained Systemscited by 38
  14. 2025Representation Learning for Tabular Data: A Comprehensive Surveycited by 36
  15. 2024Online Adaptation of Language Models with a Memory of Amortized Contextscited by 35
  16. 2024Active Evaluation Acquisition for Efficient LLM Benchmarkingcited by 35
  17. 2021The Gaussian Neural Processcited by 34
  18. 2021Context-Aware Safe Reinforcement Learning for Non-Stationary Environmentscited by 33
  19. 2023Autoregressive Conditional Neural Processescited by 33
  20. 2023End-to-End Meta-Bayesian Optimisation with Transformer Neural Processescited by 32
  21. 2024Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodingscited by 31
  22. 2021Convolutional conditional neural processes for local climate downscalingcited by 29
  23. 2021Attentive Neural Controlled Differential Equations for Time-series Classification and Forecastingcited by 29
  24. 2021Data-Driven Offline Optimization For Architecting Hardware Acceleratorscited by 29
  25. 2023Diffusion Probabilistic Fieldscited by 28
  26. 2023Transfer Learning for Bayesian Optimization: A Surveycited by 27
  27. 2024Learning to Defer to a Population: A Meta-Learning Approachcited by 27
  28. 2021NRTSI: Non-Recurrent Time Series Imputationcited by 25
  29. 2021Conditional Image Generation by Conditioning Variational Auto-Encoderscited by 25
  30. 2022Practical Conditional Neural Processes Via Tractable Dependent Predictionscited by 25
  31. 2022Variable-Input Deep Operator Networkscited by 25
  32. 2022Spectral Diffusion Processescited by 25
  33. 2021How Tight Can PAC-Bayes be in the Small Data Regime?cited by 24
  34. 2021The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learningcited by 23
  35. 2022The Neural Process Family: Survey, Applications and Perspectivescited by 23
  36. 2024Amortized Probabilistic Conditioning for Optimization, Simulation and Inferencecited by 23
  37. 2023Variational Autoencoding Neural Operatorscited by 22
  38. 2023ExPT: Synthetic Pretraining for Few-Shot Experimental Designcited by 22

50 most cited of 283 All 283 (JSON)