# A RAD approach to deep mixture models

arXiv: https://arxiv.org/abs/1903.07714 · HTML: https://mlsanity.com/paper/1903.07714/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2019.

## Built on by (most cited first, 18 of 18)

- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [A Variational Perspective on Diffusion-Based Generative Models and Score Matching](https://mlsanity.com/paper/2106.02808/) (2021)
- [Diffusion Normalizing Flow](https://mlsanity.com/paper/2110.07579/) (2021)
- [Verifying the Union of Manifolds Hypothesis for Image Data](https://mlsanity.com/paper/2207.02862/) (2022)
- [Featurized Density Ratio Estimation](https://mlsanity.com/paper/2107.02212/) (2021)
- [Go with the Flows: Mixtures of Normalizing Flows for Point Cloud Generation and Reconstruction](https://mlsanity.com/paper/2106.03135/) (2021)
- [On the expressivity of bi-Lipschitz normalizing flows](https://mlsanity.com/paper/2107.07232/) (2021)
- [Density estimation on smooth manifolds with normalizing flows](https://mlsanity.com/paper/2106.03500/) (2021)
- [Semi-Discrete Normalizing Flows through Differentiable Tessellation](https://mlsanity.com/paper/2203.06832/) (2022)
- [Out-of-Distribution Detection with Reconstruction Error and Typicality-based Penalty](https://mlsanity.com/paper/2212.12641/) (2022)
- [VQ-Flows: Vector Quantized Local Normalizing Flows](https://mlsanity.com/paper/2203.11556/) (2022)
- [Topology-Matching Normalizing Flows for Out-of-Distribution Detection in Robot Learning](https://mlsanity.com/paper/2311.06481/) (2023)
- [Knowledge Graph Embedding by Normalizing Flows](https://mlsanity.com/paper/2409.19977/) (2024)
- [Accelerated Bayesian parameter estimation and model selection for gravitational waves with normalizing flows](https://mlsanity.com/paper/2410.21076/) (2024)
- [Enhanced Importance Sampling through Latent Space Exploration in Normalizing Flows](https://mlsanity.com/paper/2501.03394/) (2025)
- [Out-of-Distribution Detection of Melanoma using Normalizing Flows](https://mlsanity.com/paper/2103.12672/) (2021)
- [Discretely Indexed Flows](https://mlsanity.com/paper/2204.01361/) (2022)

## Abstract (arXiv)

Flow based models such as Real NVP are an extremely powerful approach to density estimation. However, existing flow based models are restricted to transforming continuous densities over a continuous input space into similarly continuous distributions over continuous latent variables. This makes them poorly suited for modeling and representing discrete structures in data distributions, for example class membership or discrete symmetries. To address this difficulty, we present a normalizing flow architecture which relies on domain partitioning using locally invertible functions, and possesses both real and discrete valued latent variables. This Real and Discrete (RAD) approach retains the desirable normalizing flow properties of exact sampling, exact inference, and analytically computable probabilities, while at the same time allowing simultaneous modeling of both continuous and discrete structure in a data distribution.