mlsanityData to 14 Oct 2026

Entropy-Informed Weighting Channel Normalizing Flow for Deep Generative Models

arXiv 2407.04958 ↗First posted 6 Jul 2024cs.LG cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Normalizing Flows (NFs) are widely used in deep generative models for their exact likelihood estimation and efficient sampling. However, they require substantial memory since the latent space matches the input dimension. Multi-scale architectures address this by progressively reducing latent dimensions while preserving reversibility. Existing multi-scale architectures use simple, static channel-wise splitting, limiting expressiveness. To improve this, we introduce a regularized, feature-dependent mathttShuffle operation and integrate it into vanilla multi-scale architecture. This operation adaptively generates channel-wise weights and shuffles latent variables before splitting them. We observe that such operation guides the variables to evolve in the direction of entropy increase, hence we refer to NFs with the mathttShuffle operation as Entropy-Informed Weighting Channel Normalizing Flow (EIW-Flow). Extensive experiments on CIFAR-10, CelebA, ImageNet, and LSUN demonstrate that EIW-Flow achieves state-of-the-art density estimation and competitive sample quality for deep generative modeling, with minimal computational overhead.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 71 of its 115 references that are in the map. First posted on arXiv 6 Jul 2024.

Builds on

71 of 115 references.

References · 71

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2017Squeeze-and-Excitation Networkscited by 4,789
  5. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  7. 2018CBAM: Convolutional Block Attention Modulecited by 2,477
  8. 2017Improved Training of Wasserstein GANscited by 2,235
  9. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  10. 2016Density estimation using Real NVPcited by 2,042
  11. 2020Knowledge Distillation: A Surveycited by 1,603
  12. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
38 more
  1. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,323
  2. 2018Image Super-Resolution Using Very Deep Residual Channel Attention Networkscited by 1,264
  3. 2021Variational Diffusion Modelscited by 1,153
  4. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
  5. 2016An overview of gradient descent optimization algorithmscited by 1,033
  6. 2019An Introduction to Variational Autoencoderscited by 1,000
  7. 2016Conditional Image Generation with PixelCNN Decoderscited by 862
  8. 2016Pixel Recurrent Neural Networkscited by 816
  9. 2018Path Aggregation Network for Instance Segmentationcited by 798
  10. 2018Image Transformercited by 772
  11. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
  12. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 637
  13. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 488
  14. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 471
  15. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 417
  16. 2018Invertible Residual Networkscited by 365
  17. 2020Differentiable Augmentation for Data-Efficient GAN Trainingcited by 363
  18. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 309
  19. 2015A note on the evaluation of generative modelscited by 292
  20. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 262
  21. 2019Residual Flows for Invertible Generative Modelingcited by 257
  22. 2017Advances in Variational Inferencecited by 236
  23. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
  24. 2020BayesFlow: Learning complex stochastic models with invertible neural networkscited by 144
  25. 2017PixelSNAIL: An Improved Autoregressive Generative Modelcited by 137
  26. 2019From Variational to Deterministic Autoencoderscited by 133
  27. 2019BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modelingcited by 109
  28. 2020VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Modelscited by 104
  29. 2023Input Perturbation Reduces Exposure Bias in Diffusion Modelscited by 90
  30. 2019Preventing Posterior Collapse with delta-VAEscited by 86
  31. 2019Flow Contrastive Estimation of Energy-Based Modelscited by 75
  32. 2018Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscalingcited by 74
  33. 2020SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flowscited by 71
  34. 2020Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Modelscited by 66
  35. 2020A Contrastive Learning Approach for Training Variational Autoencoder Priorscited by 66
  36. 2018The challenge of realistic music generation: modelling raw audio at scalecited by 63
  37. 2020Dual Contradistinctive Generative Autoencodercited by 58
  38. 2021Consistency Regularization for Variational Auto-Encoderscited by 51

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026A Minimum Variance Path Principle for Accurate and Stable Score-Based Density Ratio Estimationcited by 3
  2. 2026Mitigating the Contractivity Trap in Diffusion ODEs via Stein Stabilizationcited by 3
  3. 2026Towards Disentangled Preference Optimization Dynamics: Suppress the Loser, Preserve the Winnercited by 2
  4. 2026DiFA: Inference-Time Forward-Process Alignment for Diffusion Modelscited by 1
  5. 2025Gaussian Process Tilted Nonparametric Density Estimation using Fisher Divergence Score Matchingcited by 0