mlsanityData to 7 Oct 2026

Densely connected normalizing flows

arXiv 2106.04627 ↗First posted 8 Jun 2021cs.LG cs.AI cs.CVcited by 49

Not on the map. Why?

Abstract (arXiv)

Normalizing flows are bijective mappings between inputs and latent representations with a fully factorized distribution. They are very attractive due to exact likelihood valuation and efficient sampling. However, their effective capacity is often insufficient since the bijectivity constraint limits the model width. We address this issue by incrementally padding intermediate representations with noise. We precondition the noise in accordance with previous invertible units, which we describe as cross-unit coupling. Our invertible glow-like modules increase the model expressivity by fusing a densely connected block with Nystrom self-attention. We refer to our architecture as DenseFlow since both cross-unit and intra-module couplings rely on dense connectivity. Experiments show significant improvements due to the proposed contributions and reveal state-of-the-art density estimation under moderate computing budgets.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 55 of its 80 references that are in the map. First posted on arXiv 8 Jun 2021.

Builds on

55 of 80 references.

References · 55

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  8. 2016Densely Connected Convolutional Networkscited by 6,691
  9. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  10. 2013Auto-Encoding Variational Bayescited by 3,980
  11. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  12. 2014Deep Learning Face Attributes in the Wildcited by 3,679
38 more
  1. 2017Non-local Neural Networkscited by 2,290
  2. 2017Improved Training of Wasserstein GANscited by 2,223
  3. 2015Variational Inference with Normalizing Flowscited by 2,161
  4. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  5. 2016Density estimation using Real NVPcited by 2,029
  6. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  7. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  8. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  9. 2016Pixel Recurrent Neural Networkscited by 814
  10. 2018Image Transformercited by 770
  11. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  12. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  13. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  14. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 469
  15. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 468
  16. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  17. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  18. 2018Invertible Residual Networkscited by 364
  19. 2020Differentiable Augmentation for Data-Efficient GAN Trainingcited by 361
  20. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
  21. 2019Residual Flows for Invertible Generative Modelingcited by 256
  22. 2018Neural Importance Samplingcited by 154
  23. 2017PixelSNAIL: An Improved Autoregressive Generative Modelcited by 137
  24. 2018i-RevNet: Deep Invertible Networkscited by 137
  25. 2019BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modelingcited by 109
  26. 2020VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Modelscited by 103
  27. 2019Preventing Posterior Collapse with delta-VAEscited by 86
  28. 2018Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscalingcited by 74
  29. 2020SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flowscited by 71
  30. 2020Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Modelscited by 66
  31. 2015On Markov chain Monte Carlo methods for tall datacited by 50
  32. 2021Consistency Regularization for Variational Auto-Encoderscited by 50
  33. 2019Hybrid Models with Deep and Invertible Featurescited by 41
  34. 2020VFlow: More Expressive Generative Flows with Variational Data Augmentationcited by 38
  35. 2016Towards Conceptual Compressioncited by 36
  36. 2020Wavelet Flow: Fast Training of High Resolution Normalizing Flowscited by 33
  37. 2019MaCow: Masked Convolutional Generative Flowcited by 32
  38. 2018DVAE++: Discrete Variational Autoencoders with Overlapping Transformationscited by 31

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 8; 2022: 11; 2023: 18; 2024: 7; 2025: 5.

49 papers.

Sort:

By area:

  1. 2023Consistency Modelscited by 1,778
  2. 2022The Vendi Score: A Diversity Evaluation Metric for Machine Learningcited by 299
  3. 2024Consistency Flow Matching: Defining Straight Flows with Velocity Consistencycited by 101
  4. 2021Hierarchical Transformers Are More Efficient Language Modelscited by 99
  5. 2021Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimationcited by 95
  6. 2022Generating High Fidelity Data from Low-density Regions using Diffusion Modelscited by 78
  7. 2023LAMP: Learn A Motion Pattern for Few-Shot-Based Video Generationcited by 59
  8. 2023normflows: A PyTorch Package for Normalizing Flowscited by 56
  9. 2021Stable, Fast and Accurate: Kernelized Attention with Relative Positional Encodingcited by 50
  10. 2021Resampling Base Distributions of Normalizing Flowscited by 42
  11. 2022Maximum Likelihood Training of Implicit Nonlinear Diffusion Modelscited by 42
  12. 2024Outlier detection by ensembling uncertainty with negative objectnesscited by 34
37 more
  1. 2022Efficient-VDVAE: Less is morecited by 26
  2. 2023Transformer-VQ: Linear-Time Transformers via Vector Quantizationcited by 26
  3. 2023AI-Generated Content (AIGC) for Various Data Modalities: A Surveycited by 25
  4. 2023MaskDiffusion: Boosting Text-to-Image Consistency with Conditional Maskcited by 17
  5. 2022Boomerang: Local sampling on image manifolds using diffusion modelscited by 16
  6. 2022ButterflyFlow: Building Invertible Layers with Butterfly Matricescited by 12
  7. 2024Jet: A Modern Transformer-Based Normalizing Flowcited by 12
  8. 2024ASPIRE: Iterative Amortized Posterior Inference for Bayesian Inverse Problemscited by 11
  9. 2021Dense Out-of-Distribution Detection by Robust Learning on Synthetic Negative Datacited by 10
  10. 2024Scalable Normalizing Flows Enable Boltzmann Generators for Macromoleculescited by 10
  11. 2023Precision-Recall Divergence Optimization for Generative Modeling with GANs and Normalizing Flowscited by 9
  12. 2023Hybrid Open-set Segmentation with Synthetic Negative Datacited by 8
  13. 2023LU-Net: Invertible Neural Networks Based on Matrix Factorizationcited by 8
  14. 2023Diffusion Models with Deterministic Normalizing Flow Priorscited by 7
  15. 2023Score Mismatching for Generative Modelingcited by 7
  16. 2022Invertible Monotone Operators for Normalizing Flowscited by 6
  17. 2021Bootstrap Your Flowcited by 5
  18. 2023ACT-Diffusion: Efficient Adversarial Consistency Training for One-step Diffusion Modelscited by 5
  19. 2024Entropy-Informed Weighting Channel Normalizing Flow for Deep Generative Modelscited by 5
  20. 2021Multi-Resolution Continuous Normalizing Flowscited by 4
  21. 2022Fake It Till You Make It: Towards Accurate Near-Distribution Novelty Detectioncited by 4
  22. 2022Fast Lossless Neural Compression with Integer-Only Discrete Flowscited by 4
  23. 2022Taming Normalizing Flowscited by 3
  24. 2024This Probably Looks Exactly Like That: An Invertible Prototypical Networkcited by 3
  25. 2025A Data-Driven Novelty Score for Diverse In-Vehicle Data Recordingcited by 3
  26. 2025Accelerate TarFlow Sampling with GS-Jacobi Iterationcited by 2
  27. 2022Autoregressive Generative Modeling with Noise Conditional Maximum Likelihood Estimationcited by 1
  28. 20233D GANs and Latent Space: A comprehensive surveycited by 1
  29. 2023Piecewise Normalizing Flowscited by 1
  30. 2025CLOFAI: A Dataset of Real And Fake Image Classification Tasks for Continual Learningcited by 1
  31. 2025Learning Straight Flows by Learning Curved Interpolantscited by 1
  32. 2021Attentive Contractive Flow with Lipschitz-constrained Self-Attentioncited by 0
  33. 2023Learning Stationary Markov Processes with Contrastive Adjustmentcited by 0
  34. 2023Invertible Convolution with Symmetric Paddingscited by 0
  35. 2023Real time dense anomaly detection by learning on synthetic negative datacited by 0
  36. 2023Out-of-distribution detection using normalizing flows on the data manifoldcited by 0
  37. 2025Breaking the Likelihood-Quality Trade-off in Diffusion Models by Merging Pretrained Expertscited by 0