mlsanityData to 7 Oct 2026

Residual Flows for Invertible Generative Modeling

arXiv 1906.02735 ↗First posted 6 Jun 2019stat.ML cs.LGcited by 256

Not on the map. Why?

Cited by 36 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Flow-based generative models parameterize probability distributions through an invertible transformation and can be trained by maximum likelihood. Invertible residual networks provide a flexible family of transformations where only Lipschitz conditions rather than strict architectural constraints are needed for enforcing invertibility. However, prior work trained invertible residual networks for density estimation by relying on biased log-density estimates whose bias increased with the network's expressiveness. We give a tractable unbiased estimate of the log density using a "Russian roulette" estimator, and reduce the memory required during training by using an alternative infinite series for the gradient. Furthermore, we improve invertible residual blocks by proposing the use of activation functions that avoid derivative saturation and generalizing the Lipschitz condition to induced mixed norms. The resulting approach, called Residual Flows, achieves state-of-the-art performance on density estimation amongst flow-based models, and outperforms networks that use coupling blocks at joint generative and discriminative modeling.

Key facts. Cited by 256 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 62; 2022: 41; 2023: 58; 2024: 38; 2025: 36; 2026: 21.

256 papers.

Sort:

By area:

  1. 2022Flow Matching for Generative Modelingcited by 5,426
  2. 2023Consistency Modelscited by 1,778
  3. 2023Improved Techniques for Training Consistency Modelscited by 430
  4. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  5. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  6. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Modelscited by 232
  7. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusioncited by 230
  8. 2023Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion modelscited by 199
  9. 2021Path Integral Sampler: a stochastic control approach for samplingcited by 175
  10. 2024Consistency Flow Matching: Defining Straight Flows with Velocity Consistencycited by 101
  11. 2021Independent mechanism analysis, a new concept?cited by 96
  12. 2022Poisson Flow Generative Modelscited by 95
38 more
  1. 2022GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Modelscited by 87
  2. 2025Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistencycited by 81
  3. 2021Diffusion Normalizing Flowcited by 77
  4. 2021On Training Implicit Modelscited by 73
  5. 2021Efficient and Accurate Gradients for Neural SDEscited by 68
  6. 2024One-Step Diffusion Distillation through Score Implicit Matchingcited by 68
  7. 2021Moser Flow: Divergence-based Generative Modeling on Manifoldscited by 67
  8. 2022Neural Conservation Laws: A Divergence-Free Perspectivecited by 67
  9. 2022Verifying the Union of Manifolds Hypothesis for Image Datacited by 62
  10. 2022Towards Blind Watermarking: Combining Invertible and Non-invertible Mechanismscited by 61
  11. 2023LAMP: Learn A Motion Pattern for Few-Shot-Based Video Generationcited by 59
  12. 2023normflows: A PyTorch Package for Normalizing Flowscited by 56
  13. 2024360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Modelcited by 56
  14. 2021Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networkscited by 53
  15. 2021Rectangular Flows for Manifold Learningcited by 49
  16. 2021Densely connected normalizing flowscited by 49
  17. 2022A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Modelcited by 44
  18. 2023How Smooth Is Attention?cited by 44
  19. 2021MALI: A memory efficient and reverse accurate integrator for Neural ODEscited by 42
  20. 2021Resampling Base Distributions of Normalizing Flowscited by 42
  21. 2022UVCGAN: UNet Vision Transformer cycle-consistent GAN for unpaired image-to-image translationcited by 42
  22. 2022Maximum Likelihood Training of Implicit Nonlinear Diffusion Modelscited by 42
  23. 2023Counterfactual Identifiability of Bijective Causal Modelscited by 42
  24. 2022Normalizing flow neural networks by JKO schemecited by 39
  25. 2021Trumpets: Injective Flows for Inference and Inverse Problemscited by 36
  26. 2021Stochastic Normalizing Flows for Inverse Problems: a Markov Chains Viewpointcited by 32
  27. 2023Autoregressive Conditional Neural Processescited by 32
  28. 2021Stability and Generalization of Bilevel Programming in Hyperparameter Optimizationcited by 31
  29. 2023User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical Systemscited by 30
  30. 2024Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferencescited by 30
  31. 2021Implicit Normalizing Flowscited by 28
  32. 2021JEM++: Improved Techniques for Training JEMcited by 28
  33. 2022Diagnosing and Fixing Manifold Overfitting in Deep Generative Modelscited by 28
  34. 2023Fast Inference and Update of Probabilistic Density Estimation on Trajectory Predictioncited by 27
  35. 2023Free-form Flows: Make Any Architecture a Normalizing Flowcited by 26
  36. 2024Flow Generator Matchingcited by 26
  37. 2023Cousins Of The Vendi Score: A Family Of Similarity-Based Diversity Metrics For Science And Machine Learningcited by 24
  38. 2021Generalized Normalizing Flows via Markov Chainscited by 23

50 most cited of 256 All 256 (JSON)