# Categorical Normalizing Flows via Continuous Transformations

arXiv: https://arxiv.org/abs/2006.09790 · HTML: https://mlsanity.com/paper/2006.09790/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2020.

## Built on by (most cited first, 30 of 37)

- [Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions](https://mlsanity.com/paper/2102.05379/) (2021)
- [DiGress: Discrete Denoising diffusion for graph generation](https://mlsanity.com/paper/2209.14734/) (2022)
- [Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning](https://mlsanity.com/paper/2208.04202/) (2022)
- [GraphDF: A Discrete Flow Model for Molecular Graph Generation](https://mlsanity.com/paper/2102.01189/) (2021)
- [GraphEBM: Molecular Graph Generation with Energy-Based Models](https://mlsanity.com/paper/2102.00546/) (2021)
- [Variational Flow Matching for Graph Generation](https://mlsanity.com/paper/2406.04843/) (2024)
- [Efficient and Degree-Guided Graph Generation via Discrete Diffusion Modeling](https://mlsanity.com/paper/2305.04111/) (2023)
- [Keeping it Simple: Language Models can learn Complex Molecular Distributions](https://mlsanity.com/paper/2112.03041/) (2021)
- [Conditional Diffusion Based on Discrete Graph Structures for Molecular Graph Generation](https://mlsanity.com/paper/2301.00427/) (2023)
- [Genetic algorithms are strong baselines for molecule generation](https://mlsanity.com/paper/2310.09267/) (2023)
- [SwinGNN: Rethinking Permutation Invariance in Diffusion Models for Graph Generation](https://mlsanity.com/paper/2307.01646/) (2023)
- [Learning Joint 2D & 3D Diffusion Models for Complete Molecule Generation](https://mlsanity.com/paper/2305.12347/) (2023)
- [iFlow: Numerically Invertible Flows for Efficient Lossless Compression via a Uniform Coder](https://mlsanity.com/paper/2111.00965/) (2021)
- [Generative Modelling of Structurally Constrained Graphs](https://mlsanity.com/paper/2406.17341/) (2024)
- [Target-aware Molecular Graph Generation](https://mlsanity.com/paper/2202.04829/) (2022)
- [NVDiff: Graph Generation through the Diffusion of Node Vectors](https://mlsanity.com/paper/2211.10794/) (2022)
- [Semi-Discrete Normalizing Flows through Differentiable Tessellation](https://mlsanity.com/paper/2203.06832/) (2022)
- [Modular Flows: Differential Molecular Generation](https://mlsanity.com/paper/2210.06032/) (2022)
- [Generative Modeling of Discrete Joint Distributions by E-Geodesic Flow Matching on Assignment Manifolds](https://mlsanity.com/paper/2402.07846/) (2024)
- [Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows](https://mlsanity.com/paper/2507.00425/) (2025)
- [MolHF: A Hierarchical Normalizing Flow for Molecular Graph Generation](https://mlsanity.com/paper/2305.08457/) (2023)
- [Manifold Density Estimation via Generalized Dequantization](https://mlsanity.com/paper/2102.07143/) (2021)
- [Diffusing Gaussian Mixtures for Generating Categorical Data](https://mlsanity.com/paper/2303.04635/) (2023)
- [Regressor-free Molecule Generation to Support Drug Response Prediction](https://mlsanity.com/paper/2405.14536/) (2024)
- [Generating Directed Graphs with Dual Attention and Asymmetric Encoding](https://mlsanity.com/paper/2506.16404/) (2025)
- [Simplex-to-Euclidean Bijections for Categorical Flow Matching](https://mlsanity.com/paper/2510.27480/) (2025)
- [Flowette: Flow Matching with Graphette Priors for Graph Generation](https://mlsanity.com/paper/2602.23566/) (2026)
- [Mixed Variational Flows for Discrete Variables](https://mlsanity.com/paper/2308.15613/) (2023)
- [Molecular Generative Adversarial Network with Multi-Property Optimization](https://mlsanity.com/paper/2404.00081/) (2024)
- [Targeting the partition function of chemically disordered materials with a generative approach based on inverse variational autoencoders](https://mlsanity.com/paper/2408.14928/) (2024)

## Abstract (arXiv)

Despite their popularity, to date, the application of normalizing flows on categorical data stays limited. The current practice of using dequantization to map discrete data to a continuous space is inapplicable as categorical data has no intrinsic order. Instead, categorical data have complex and latent relations that must be inferred, like the synonymy between words. In this paper, we investigate Categorical Normalizing Flows, that is normalizing flows for categorical data. By casting the encoding of categorical data in continuous space as a variational inference problem, we jointly optimize the continuous representation and the model likelihood. Using a factorized decoder, we introduce an inductive bias to model any interactions in the normalizing flow. As a consequence, we do not only simplify the optimization compared to having a joint decoder, but also make it possible to scale up to a large number of categories that is currently impossible with discrete normalizing flows. Based on Categorical Normalizing Flows, we propose GraphCNF a permutation-invariant generative model on graphs. GraphCNF implements a three step approach modeling the nodes, edges and adjacency matrix stepwise to increase efficiency. On molecule generation, GraphCNF outperforms both one-shot and autoregressive flow-based state-of-the-art.