# On the Conditioning Consistency Gap in Conditional Neural Processes

arXiv: https://arxiv.org/abs/2604.19312 · HTML: https://mlsanity.com/paper/2604.19312/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 13 references that are in the map.
- First posted on arXiv 21 Apr 2026.

## Builds on (most cited first, 11 of 11)

- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [Convolutional Conditional Neural Processes](https://mlsanity.com/paper/1910.13556/) (2019)
- [Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling](https://mlsanity.com/paper/2207.04179/) (2022)
- [PACOH: Bayes-Optimal Meta-Learning with PAC-Guarantees](https://mlsanity.com/paper/2002.05551/) (2020)
- [Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory](https://mlsanity.com/paper/1711.01244/) (2017)
- [Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processes](https://mlsanity.com/paper/2007.01332/) (2020)
- [A PAC-Bayesian bound for Lifelong Learning](https://mlsanity.com/paper/1311.2838/) (2013)
- [Neural Diffusion Processes](https://mlsanity.com/paper/2206.03992/) (2022)
- [Information-Theoretic Generalization Bounds for Meta-Learning and Applications](https://mlsanity.com/paper/2005.04372/) (2020)
- [Flow Matching Neural Processes](https://mlsanity.com/paper/2512.23853/) (2025)

## Built on by (most cited first, 3 of 3)

- [Do Tabular Foundation Models Agree with Themselves?](https://mlsanity.com/paper/2608.06004/) (2026)
- [Three Costs of Amortizing Gaussian Process Inference with Neural Processes](https://mlsanity.com/paper/2605.21798/) (2026)
- [Characterizing the Representational Capacity of Neural Processes](https://mlsanity.com/paper/2605.24210/) (2026)

## Abstract (arXiv)

Neural processes are meta-learning models that map context sets to predictive distributions. While inspired by stochastic processes, NPs do not generally satisfy the Kolmogorov consistency conditions required to define a valid stochastic process. This inconsistency is widely acknowledged but poorly understood. Practitioners note that NPs work well despite the violation, without quantifying what this means. We address this gap by defining the conditioning consistency gap, a KL divergence measuring how much a conditional neural process's (CNP) predictions change when a point is added to the context versus conditioned upon. Our main results show that for CNPs with bounded encoders and Lipschitz decoders, the consistency gap is O(1/n2) in context size n, and that this rate is tight. These bounds establish the precise sense in which CNPs approximate valid stochastic processes. The inconsistency is negligible for moderate context sizes but can be significant in the few-shot regime.