mlsanityData to 7 Oct 2026

Discrete Flows: Invertible Generative Models of Discrete Data

arXiv 1905.10347 ↗First posted 24 May 2019cs.LG stat.MLcited by 68

Not on the map. Why?

Abstract (arXiv)

While normalizing flows have led to significant advances in modeling high-dimensional continuous distributions, their applicability to discrete distributions remains unknown. In this paper, we show that flows can in fact be extended to discrete events---and under a simple change-of-variables formula not requiring log-determinant-Jacobian computations. Discrete flows have numerous applications. We consider two flow architectures: discrete autoregressive flows that enable bidirectionality, allowing, for example, tokens in text to depend on both left-to-right and right-to-left contexts in an exact language model; and discrete bipartite flows that enable efficient non-autoregressive generation as in RealNVP. Empirically, we find that discrete autoregressive flows outperform autoregressive baselines on synthetic discrete distributions, an addition task, and Potts models; and bipartite flows can obtain competitive performance with autoregressive baselines on character-level language modeling for Penn Tree Bank and text8.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 13; 2023: 13; 2024: 10; 2025: 8; 2026: 7.

68 papers.

Sort:

By area:

  1. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  2. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  3. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  4. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  5. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  6. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  7. 2021GraphDF: A Discrete Flow Model for Molecular Graph Generationcited by 177
  8. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  9. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 144
  10. 2024Energy-Based Diffusion Language Models for Text Generationcited by 86
  11. 2022An Introduction to Neural Data Compressioncited by 78
  12. 2023Bayesian Flow Networkscited by 62
38 more
  1. 2021Variational Causal Networks: Approximate Bayesian Inference over Causal Structurescited by 43
  2. 2024Categorical Flow Matching on Statistical Manifoldscited by 43
  3. 2021Fair Normalizing Flowscited by 38
  4. 2021Copula Flows for Synthetic Data Generationcited by 35
  5. 2025LEAPS: A discrete neural sampler via locally equivariant networkscited by 30
  6. 2025Continuous Diffusion Model for Language Modelingcited by 29
  7. 2022Controllable Text Generation via Probability Density Estimation in the Latent Spacecited by 27
  8. 2023TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Seriescited by 25
  9. 2021Contingencies from Observations: Tractable Contingency Planning with Learned Behavior Modelscited by 22
  10. 2021MolGrow: A Graph Normalizing Flow for Hierarchical Molecular Generationcited by 22
  11. 2022Sparse Probabilistic Circuits via Pruning and Growingcited by 17
  12. 2025CtrlDiff: Boosting Large Diffusion Language Models with Dynamic Block Prediction and Controllable Generationcited by 17
  13. 2022Personalized Public Policy Analysis in Social Sciences using Causal-Graphical Normalizing Flowscited by 16
  14. 2021Out-of-Distribution Detection for Medical Applications: Guidelines for Practical Evaluationcited by 15
  15. 2023Learning Causally Disentangled Representations via the Principle of Independent Causal Mechanismscited by 14
  16. 2021Memory-Efficient Differentiable Transformer Architecture Searchcited by 13
  17. 2022Semi-Discrete Normalizing Flows through Differentiable Tessellationcited by 11
  18. 2021Partition and Code: learning how to compress graphscited by 10
  19. 2024Generative Modeling of Discrete Joint Distributions by E-Geodesic Flow Matching on Assignment Manifoldscited by 9
  20. 2023Probabilistic Integral Circuitscited by 8
  21. 2022Gradient Estimation for Binary Latent Variables via Gradient Variance Clippingcited by 7
  22. 2023Learning a Deep Color Difference Metric for Photographic Imagescited by 6
  23. 2021Normalizing Flows for Knockoff-free Controlled Feature Selectioncited by 5
  24. 2021Counterfactual Maximum Likelihood Estimation for Training Deep Networkscited by 5
  25. 2023Modeling Uncertainty in Personalized Emotion Prediction with Normalizing Flowscited by 5
  26. 2025Non-Markovian Discrete Diffusion with Causal Language Modelscited by 5
  27. 2022Flow-Adapter Architecture for Unsupervised Machine Translationcited by 4
  28. 2023Machine Learning and the Future of Bayesian Computationcited by 4
  29. 2023Generative Marginalization Modelscited by 4
  30. 2024ContextFlow++: Generalist-Specialist Flow-based Generative Models with Mixed-Variable Context Encodingcited by 3
  31. 2025ShortListing Model: A Streamlined SimplexDiffusion for Discrete Variable Generationcited by 3
  32. 2026Discrete Stochastic Localization for Non-autoregressive Generationcited by 3
  33. 2026Generalized Discrete Diffusion from Snapshotscited by 3
  34. 2021Improving Bridge estimators via f-GANcited by 2
  35. 2021Discrete Denoising Flowscited by 2
  36. 2023Mixed Variational Flows for Discrete Variablescited by 2
  37. 2025In-Situ Tweedie Discrete Diffusion Modelscited by 2
  38. 2026Do Diffusion Models Dream of Electric Planes? Discrete and Continuous Simulation-Based Inference for Aircraft Designcited by 2

50 most cited of 68 All 68 (JSON)