mlsanityData to 7 Oct 2026

Latent Normalizing Flows for Discrete Sequences

arXiv 1901.10548 ↗First posted 29 Jan 2019stat.ML cs.CL cs.LGcited by 66

Not on the map. Why?

Abstract (arXiv)

Normalizing flows are a powerful class of generative models for continuous random variables, showing both strong model flexibility and the potential for non-autoregressive generation. These benefits are also desired when modeling discrete random variables such as text, but directly applying normalizing flows to discrete sequences poses significant additional challenges. We propose a VAE-based generative model which jointly learns a normalizing flow-based distribution in the latent space and a stochastic mapping to an observed discrete space. In this setting, we find that it is crucial for the flow-based distribution to be highly multimodal. To capture this property, we propose several normalizing flow architectures to maximize model flexibility. Experiments consider common discrete sequence tasks of character-level language modeling and polyphonic music generation. Our results indicate that an autoregressive flow-based model can match the performance of a comparable autoregressive baseline, and a non-autoregressive flow-based model can improve generation speed with a penalty to performance.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 19; 2023: 11; 2024: 10; 2025: 10; 2026: 6.

66 papers.

Sort:

By area:

  1. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  2. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  3. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  4. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  5. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  6. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  7. 2022A Survey on Generative Diffusion Modelcited by 238
  8. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  9. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 144
  10. 2024Energy-Based Diffusion Language Models for Text Generationcited by 86
  11. 2023Bayesian Flow Networkscited by 62
  12. 2021Copula Flows for Synthetic Data Generationcited by 35
38 more
  1. 2022Universal approximation property of invertible neural networkscited by 30
  2. 2025Continuous Diffusion Model for Language Modelingcited by 29
  3. 2022Controllable Text Generation via Probability Density Estimation in the Latent Spacecited by 27
  4. 2023TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Seriescited by 25
  5. 2024CaRiNG: Learning Temporal Causal Representation under Non-Invertible Generation Processcited by 20
  6. 2025Latent Bayesian Optimization via Autoregressive Normalizing Flowscited by 19
  7. 2024On the Universality of Volume-Preserving and Coupling-Based Normalizing Flowscited by 18
  8. 2022Sparse Probabilistic Circuits via Pruning and Growingcited by 17
  9. 2025CtrlDiff: Boosting Large Diffusion Language Models with Dynamic Block Prediction and Controllable Generationcited by 17
  10. 2022Personalized Public Policy Analysis in Social Sciences using Causal-Graphical Normalizing Flowscited by 16
  11. 2022Semi-Discrete Normalizing Flows through Differentiable Tessellationcited by 11
  12. 2021PassFlow: Guessing Passwords with Generative Flowscited by 10
  13. 2022Conditional Independence Testing via Latent Representation Learningcited by 9
  14. 2025Scaling Probabilistic Circuits via Monarch Matricescited by 9
  15. 2022Low-Rank Constraints for Fast Inference in Structured Modelscited by 8
  16. 2022TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speedcited by 8
  17. 2023Probabilistic Integral Circuitscited by 8
  18. 2025Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flowscited by 8
  19. 2024Deep Learning With DAGscited by 7
  20. 2022Invertible Neural Networks for Graph Predictioncited by 6
  21. 2022Whitening Convergence Rate of Coupling-based Normalizing Flowscited by 6
  22. 2024Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelinescited by 6
  23. 2025Learning Flexible Forward Trajectories for Masked Molecular Diffusioncited by 6
  24. 2022Better Language Model with Hypernym Class Predictioncited by 5
  25. 2022AUTM Flow: Atomic Unrestricted Time Machine for Monotonic Normalizing Flowscited by 5
  26. 2025Non-Markovian Discrete Diffusion with Causal Language Modelscited by 5
  27. 2021Uncertainty quantification for ptychography using normalizing flowscited by 4
  28. 2021Funnels: Exact maximum likelihood with dimensionality reductioncited by 4
  29. 2022Flow-Adapter Architecture for Unsupervised Machine Translationcited by 4
  30. 2023Diffusing Gaussian Mixtures for Generating Categorical Datacited by 4
  31. 2023Machine Learning and the Future of Bayesian Computationcited by 4
  32. 2023A Flow-Based Generative Model for Rare-Event Simulationcited by 4
  33. 2026Latent Reasoning with Normalizing Flowscited by 4
  34. 2021Flow-based Spatio-Temporal Structured Prediction of Motion Dynamicscited by 3
  35. 2024Neural Fourier Modelling: A Highly Compact Approach to Time-Series Analysiscited by 3
  36. 2026Discrete Stochastic Localization for Non-autoregressive Generationcited by 3
  37. 2026Generalized Discrete Diffusion from Snapshotscited by 3
  38. 2023Mixed Variational Flows for Discrete Variablescited by 2

50 most cited of 66 All 66 (JSON)