# Adversarially Contrastive Estimation of Conditional Neural Processes

arXiv: https://arxiv.org/abs/2303.13004 · HTML: https://mlsanity.com/paper/2303.13004/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 49 references that are in the map.
- First posted on arXiv 23 Mar 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications](https://mlsanity.com/paper/2001.06937/) (2020)
- [Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model](https://mlsanity.com/paper/1904.09770/) (2019)
- [Convolutional Conditional Neural Processes](https://mlsanity.com/paper/1910.13556/) (2019)
- [Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling](https://mlsanity.com/paper/2207.04179/) (2022)
- [Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency](https://mlsanity.com/paper/1809.01812/) (2018)
- [Learning Latent Space Energy-Based Prior Model](https://mlsanity.com/paper/2006.08205/) (2020)
- [VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models](https://mlsanity.com/paper/2010.00654/) (2020)
- [Taming VAEs](https://mlsanity.com/paper/1810.00597/) (2018)
- [Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processes](https://mlsanity.com/paper/2007.01332/) (2020)
- [Flow Contrastive Estimation of Energy-Based Models](https://mlsanity.com/paper/1912.00589/) (2019)
- [A Contrastive Learning Approach for Training Variational Autoencoder Priors](https://mlsanity.com/paper/2010.02917/) (2020)
- [Simple and Effective VAE Training with Calibrated Decoders](https://mlsanity.com/paper/2006.13202/) (2020)
- [Neural ODE Processes](https://mlsanity.com/paper/2103.12413/) (2021)
- [Doubly Stochastic Variational Inference for Neural Processes with Hierarchical Latent Variables](https://mlsanity.com/paper/2008.09469/) (2020)
- [Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processes](https://mlsanity.com/paper/2011.12916/) (2020)
- [Practical Conditional Neural Processes Via Tractable Dependent Predictions](https://mlsanity.com/paper/2203.08775/) (2022)
- [Leveraging the Exact Likelihood of Deep Latent Variable Models](https://mlsanity.com/paper/1802.04826/) (2018)
- [Adversarial Contrastive Estimation](https://mlsanity.com/paper/1805.03642/) (2018)
- [Function Contrastive Learning of Transferable Meta-Representations](https://mlsanity.com/paper/2010.07093/) (2020)
- [On Contrastive Representations of Stochastic Processes](https://mlsanity.com/paper/2106.10052/) (2021)
- [The Optimal Noise in Noise-Contrastive Learning Is Not What You Think](https://mlsanity.com/paper/2203.01110/) (2022)
- [Contrastive Conditional Neural Processes](https://mlsanity.com/paper/2203.03978/) (2022)

## Built on by (most cited first, 1 of 1)

- [Neural Network Reprogrammability: A Unified Theme on Model Reprogramming, Prompt Tuning, and Prompt Instruction](https://mlsanity.com/paper/2506.04650/) (2025)

## Abstract (arXiv)

Conditional Neural Processes~(CNPs) formulate distributions over functions and generate function observations with exact conditional likelihoods. CNPs, however, have limited expressivity for high-dimensional observations, since their predictive distribution is factorized into a product of unconstrained (typically) Gaussian outputs. Previously, this could be handled using latent variables or autoregressive likelihood, but at the expense of intractable training and quadratically increased complexity. Instead, we propose calibrating CNPs with an adversarial training scheme besides regular maximum likelihood estimates. Specifically, we train an energy-based model (EBM) with noise contrastive estimation, which enforces EBM to identify true observations from the generations of CNP. In this way, CNP must generate predictions closer to the ground-truth to fool EBM, instead of merely optimizing with respect to the fixed-form likelihood. From generative function reconstruction to downstream regression and classification tasks, we demonstrate that our method fits mainstream CNP members, showing effectiveness when unconstrained Gaussian likelihood is defined, requiring minimal computation overhead while preserving foundation properties of CNPs.