# Amortized Probabilistic Conditioning for Optimization, Simulation and Inference

arXiv: https://arxiv.org/abs/2410.15320 · HTML: https://mlsanity.com/paper/2410.15320/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 85 references that are in the map.
- First posted on arXiv 20 Oct 2024.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [Scalable Variational Gaussian Process Classification](https://mlsanity.com/paper/1411.2005/) (2014)
- [Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows](https://mlsanity.com/paper/1805.07226/) (2018)
- [Automatic Posterior Transformation for Likelihood-Free Inference](https://mlsanity.com/paper/1905.07488/) (2019)
- [Max-value Entropy Search for Efficient Bayesian Optimization](https://mlsanity.com/paper/1703.01968/) (2017)
- [Predictive Entropy Search for Efficient Global Optimization of Black-box Functions](https://mlsanity.com/paper/1406.2541/) (2014)
- [Flexible statistical inference for mechanistic models of neural dynamics](https://mlsanity.com/paper/1711.01861/) (2017)
- [Benchmarking Simulation-Based Inference](https://mlsanity.com/paper/2101.04653/) (2021)
- [Convolutional Conditional Neural Processes](https://mlsanity.com/paper/1910.13556/) (2019)
- [Large Language Models to Enhance Bayesian Optimization](https://mlsanity.com/paper/2402.03921/) (2024)
- [Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling](https://mlsanity.com/paper/2207.04179/) (2022)
- [Validating Bayesian Inference Algorithms with Simulation-Based Calibration](https://mlsanity.com/paper/1804.06788/) (2018)
- [Janossy Pooling: Learning Deep Permutation-Invariant Functions for Variable-Size Inputs](https://mlsanity.com/paper/1811.01900/) (2018)
- [Neural Autoregressive Distribution Estimation](https://mlsanity.com/paper/1605.02226/) (2016)
- [Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processes](https://mlsanity.com/paper/2007.01332/) (2020)
- [All-in-one simulation-based inference](https://mlsanity.com/paper/2404.09636/) (2024)

## Built on by (most cited first, 23 of 23)

- [Amortized Bayesian Experimental Design for Decision-Making](https://mlsanity.com/paper/2411.02064/) (2024)
- [Distribution Transformers: Fast Approximate Bayesian Inference With On-The-Fly Prior Adaptation](https://mlsanity.com/paper/2502.02463/) (2025)
- [Solving Empirical Bayes via Transformers](https://mlsanity.com/paper/2502.09844/) (2025)
- [PABBO: Preferential Amortized Black-Box Optimization](https://mlsanity.com/paper/2503.00924/) (2025)
- [ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisition](https://mlsanity.com/paper/2506.07259/) (2025)
- [In-Context Multi-Objective Optimization](https://mlsanity.com/paper/2512.11114/) (2025)
- [FNOPE: Simulation-based inference on function spaces with Fourier Neural Operators](https://mlsanity.com/paper/2505.22573/) (2025)
- [Efficient Autoregressive Inference for Transformer Probabilistic Models](https://mlsanity.com/paper/2510.09477/) (2025)
- [JADAI: Jointly Amortizing Adaptive Design and Bayesian Inference](https://mlsanity.com/paper/2512.22999/) (2025)
- [Meta-Statistical Learning: Supervised Learning of Statistical Estimators](https://mlsanity.com/paper/2502.12088/) (2025)
- [PriorGuide: Test-Time Prior Adaptation for Simulation-Based Inference](https://mlsanity.com/paper/2510.13763/) (2025)
- [Constrained Bayesian Experimental Design via Online Planning](https://mlsanity.com/paper/2605.26990/) (2026)
- [α-PFN: Fast Entropy Search via In-Context Learning](https://mlsanity.com/paper/2606.07134/) (2026)
- [Efficient Adaptive Data Acquisition via Pretrained Belief Representations](https://mlsanity.com/paper/2606.25197/) (2026)
- [MIST: Mutual Information Estimation Via Supervised Training](https://mlsanity.com/paper/2511.18945/) (2025)
- [A Review of Diffusion-based Simulation-Based Inference: Foundations and Applications in Non-Ideal Data Scenarios](https://mlsanity.com/paper/2512.23748/) (2025)
- [Multi-Task Bayesian In-Context Learning](https://mlsanity.com/paper/2606.20538/) (2026)
- [Guidance for Prior Change via Density Ratio Estimation](https://mlsanity.com/paper/2608.21729/) (2026)
- [Direct Regret Optimization in Bayesian Optimization](https://mlsanity.com/paper/2507.06529/) (2025)
- [Neural Bridge Processes](https://mlsanity.com/paper/2508.07220/) (2025)
- [Amortising Inference and Meta-Learning Priors in Neural Networks](https://mlsanity.com/paper/2602.08782/) (2026)
- [Amortized Factor Inference Networks for Posterior Inference](https://mlsanity.com/paper/2605.26419/) (2026)
- [In-Context Batch Black-box Optimization via Flow Matching](https://mlsanity.com/paper/2610.20138/) (2026)

## Abstract (arXiv)

Amortized meta-learning methods based on pre-training have propelled fields like natural language processing and vision. Transformer-based neural processes and their variants are leading models for probabilistic meta-learning with a tractable objective. Often trained on synthetic data, these models implicitly capture essential latent information in the data-generation process. However, existing methods do not allow users to flexibly inject (condition on) and extract (predict) this probabilistic latent information at runtime, which is key to many tasks. We introduce the Amortized Conditioning Engine (ACE), a new transformer-based meta-learning model that explicitly represents latent variables of interest. ACE affords conditioning on both observed data and interpretable latent variables, the inclusion of priors at runtime, and outputs predictive distributions for discrete and continuous data and latents. We show ACE's modeling flexibility and performance in diverse tasks such as image completion and classification, Bayesian optimization, and simulation-based inference.