# Variational Transdimensional Inference

arXiv: https://arxiv.org/abs/2506.04749 · HTML: https://mlsanity.com/paper/2506.04749/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 55 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Builds on (most cited first, 26 of 26)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [MADE: Masked Autoencoder for Distribution Estimation](https://mlsanity.com/paper/1502.03509/) (2015)
- [Monte Carlo Gradient Estimation in Machine Learning](https://mlsanity.com/paper/1906.10652/) (2019)
- [Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows](https://mlsanity.com/paper/1805.07226/) (2018)
- [Learning Likelihoods with Conditional Normalizing Flows](https://mlsanity.com/paper/1912.00042/) (2019)
- [DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization](https://mlsanity.com/paper/2209.08037/) (2022)
- [DiBS: Differentiable Bayesian Structure Learning](https://mlsanity.com/paper/2105.11839/) (2021)
- [Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Network](https://mlsanity.com/paper/2305.19366/) (2023)
- [Neural Methods for Amortized Inference](https://mlsanity.com/paper/2404.12484/) (2024)
- [Automated Variational Inference in Probabilistic Programming](https://mlsanity.com/paper/1301.1299/) (2013)
- [Flow Models for Arbitrary Conditional Likelihoods](https://mlsanity.com/paper/1909.06319/) (2019)
- [Holes in Bayesian Statistics](https://mlsanity.com/paper/2002.06467/) (2020)
- [Variational Search Distributions](https://mlsanity.com/paper/2409.06142/) (2024)
- [Transport Reversible Jump Proposals](https://mlsanity.com/paper/2210.12572/) (2022)
- [Comparison of Affine and Rational Quadratic Spline Coupling and Autoregressive Flows through Robust Statistical Tests](https://mlsanity.com/paper/2302.12024/) (2023)
- [Sequential Monte Carlo with transformations](https://mlsanity.com/paper/1612.06468/) (2016)
- [Simulations evaluating resampling methods for causal discovery: ensemble performance and calibration](https://mlsanity.com/paper/1910.02047/) (2019)
- [Mixed Variational Flows for Discrete Variables](https://mlsanity.com/paper/2308.15613/) (2023)

## Built on by (most cited first, 2 of 2)

- [Mixed neural posterior estimation for simulators with discrete and continuous parameters](https://mlsanity.com/paper/2605.13551/) (2026)
- [Structured Dimension-Matched Joint Variational Transdimensional Inference](https://mlsanity.com/paper/2608.05607/) (2026)

## Abstract (arXiv)

The expressiveness of flow-based models combined with stochastic variational inference (SVI) has expanded the application of optimization-based Bayesian inference to highly complex problems. However, despite the importance of multi-model Bayesian inference for problems defined on a transdimensional joint model and parameter space, such as Bayesian structure learning and model selection, flow-based SVI has been limited to problems defined on a fixed-dimensional parameter space. We introduce CoSMIC, normalizing flows (COntextually-Specified Masking for Identity-mapped Components), an extension to neural autoregressive conditional normalizing flow architectures that enables use of a single flow-based variational density for inference over a transdimensional (multi-model) conditional target distribution. We propose a combined stochastic variational transdimensional inference (VTI) approach to training CoSMIC, flows using ideas from Bayesian optimization and Monte Carlo gradient estimation. Numerical experiments show the performance of VTI on challenging problems that scale to high-cardinality model spaces.