mlsanityData to 14 Oct 2026

R'enyi Neural Processes

arXiv 2405.15991 ↗First posted 25 May 2024cs.LG cs.AI stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

Neural Processes (NPs) are deep probabilistic models that represent stochastic processes by conditioning their prior distributions on a set of context points. Despite their advantages in uncertainty estimation for complex distributions, NPs enforce parameterization coupling between the conditional prior model and the posterior model. We show that this coupling amounts to prior misspecification and revisit the NP objective to address this issue. More specifically, we propose R'enyi Neural Processes (RNP), a method that replaces the standard KL divergence with the R'enyi divergence, dampening the effects of the misspecified prior during posterior updates. We validate our approach across multiple benchmarks including regression and image inpainting tasks, and show significant performance improvements of RNPs in real-world problems. Our extensive experiments show consistently better log-likelihoods over state-of-the-art NP models.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 51 of its 67 references that are in the map. First posted on arXiv 25 May 2024.

Builds on

51 of 67 references.

References · 51

  1. 2017Deep Setscited by 1,421
  2. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  3. 2015Generating Sentences from a Continuous Spacecited by 612
  4. 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
  5. 2015Variational Dropout and the Local Reparameterization Trickcited by 490
  6. 2018Conditional Neural Processescited by 426
  7. 2018Neural Processescited by 311
  8. 2019Attentive Neural Processescited by 283
  9. 2017Advances in Variational Inferencecited by 236
  10. 2018Disentangling Adversarial Robustness and Generalizationcited by 172
  11. 2019Convolutional Conditional Neural Processescited by 142
  12. 2019Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processescited by 131
38 more
  1. 2022Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modelingcited by 127
  2. 2016R'enyi Divergence Variational Inferencecited by 104
  3. 2020Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processescited by 76
  4. 2019Generalized Variational Inference: Three arguments for deriving new Posteriorscited by 57
  5. 2020Simple and Effective VAE Training with Calibrated Decoderscited by 57
  6. 2022Neural Diffusion Processescited by 50
  7. 2023Learning Robust Statistics for Simulation-based Inference under Model Misspecificationcited by 48
  8. 2022Investigating the Impact of Model Misspecification in Neural Simulation-based Inferencecited by 45
  9. 2018Variational Implicit Processescited by 41
  10. 2023Tractable Function-Space Variational Inference in Bayesian Neural Networkscited by 41
  11. 2020Bootstrapping Neural Processescited by 40
  12. 2020Doubly Stochastic Variational Inference for Neural Processes with Hierarchical Latent Variablescited by 33
  13. 2023Autoregressive Conditional Neural Processescited by 33
  14. 2021Task-adaptive Neural Process for User Cold-Start Recommendationcited by 32
  15. 2023End-to-End Meta-Bayesian Optimisation with Transformer Neural Processescited by 32
  16. 2020f-Divergence Variational Inferencecited by 28
  17. 2021A Bit More Bayesian: Domain-Invariant Learning with Uncertaintycited by 27
  18. 2021Bayesian Inference with Certifiable Adversarial Robustnesscited by 26
  19. 2022Practical Conditional Neural Processes Via Tractable Dependent Predictionscited by 25
  20. 2022The Neural Process Family: Survey, Applications and Perspectivescited by 23
  21. 2020Posterior Differential Regularization with f-divergence for Improving Model Robustnesscited by 20
  22. 2023Normalizing Flow-based Neural Process for Few-Shot Knowledge Graph Completioncited by 19
  23. 2023Martingale Posterior Neural Processescited by 16
  24. 2017Variational Inference based on Robust Divergencescited by 15
  25. 2022Alpha-divergence Variational Inference Meets Importance Weighted Auto-Encoders: Methodology and Asymptoticscited by 15
  26. 2023NPCL: Neural Processes for Uncertainty-Aware Continual Learningcited by 14
  27. 2021Function-space Inference with Sparse Implicit Processescited by 13
  28. 2023Practical Equivariances via Relational Conditional Neural Processescited by 13
  29. 2018Alpha-Beta Divergence For Variational Inferencecited by 12
  30. 2023Versatile Neural Processes for Learning Implicit Neural Representationscited by 11
  31. 2023Beyond Unimodal: Generalising Neural Processes for Multimodal Uncertainty Estimationcited by 11
  32. 2023Precision-Recall Divergence Optimization for Generative Modeling with GANs and Normalizing Flowscited by 9
  33. 2022RenyiCL: Contrastive Representation Learning with Skew Renyi Divergencecited by 8
  34. 2023Random Grid Neural Processes for Parametric Partial Differential Equationscited by 8
  35. 2023Episodic Multi-Task Learning with Heterogeneous Neural Processescited by 8
  36. 2021Multi-Task Neural Processescited by 7
  37. 2023A Tutorial on Parametric Variational Inferencecited by 4
  38. 2021Global Convolutional Neural Processescited by 3

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Spectral Transformer Neural Processescited by 0