mlsanityData to 14 Oct 2026

Invertible DenseNets with Concatenated LipSwish

arXiv 2102.02694 ↗First posted 4 Feb 2021stat.ML cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

We introduce Invertible Dense Networks (i-DenseNets), a more parameter efficient extension of Residual Flows. The method relies on an analysis of the Lipschitz continuity of the concatenation in DenseNets, where we enforce invertibility of the network by satisfying the Lipschitz constant. Furthermore, we propose a learnable weighted concatenation, which not only improves the model performance but also indicates the importance of the concatenated weighted representation. Additionally, we introduce the Concatenated LipSwish as activation function, for which we show how to enforce the Lipschitz condition and which boosts performance. The new architecture, i-DenseNet, out-performs Residual Flow and other flow-based models on density estimation evaluated in bits per dimension, where we utilize an equal parameter budget. Moreover, we show that the proposed model out-performs Residual Flows when trained as a hybrid model where the model is both a generative and a discriminative model.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 30 of its 42 references that are in the map. First posted on arXiv 4 Feb 2021.

Builds on

30 of 42 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Densely Connected Convolutional Networkscited by 6,720
  3. 2015Variational Inference with Normalizing Flowscited by 2,170
  4. 2016Density estimation using Real NVPcited by 2,042
  5. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  6. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  7. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  8. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
  9. 2016Pixel Recurrent Neural Networkscited by 816
  10. 2017Masked Autoregressive Flow for Density Estimationcited by 684
  11. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
  12. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
18 more
  1. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
  2. 2018Invertible Residual Networkscited by 365
  3. 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 314
  4. 2015A note on the evaluation of generative modelscited by 292
  5. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 262
  6. 2019Residual Flows for Invertible Generative Modelingcited by 257
  7. 2018Sorting out Lipschitz function approximationcited by 213
  8. 2019Preventing Gradient Attenuation in Lipschitz Constrained Convolutional Networkscited by 87
  9. 2016Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Unitscited by 86
  10. 2018Sylvester Normalizing Flows for Variational Inferencecited by 72
  11. 2018Excessive Invariance Causes Adversarial Vulnerabilitycited by 66
  12. 2019Block Neural Autoregressive Flowcited by 62
  13. 2016Improving Variational Auto-Encoders using Householder Flowcited by 42
  14. 2019Hybrid Models with Deep and Invertible Featurescited by 41
  15. 2019Understanding the Limitations of Conditional Generative Modelscited by 28
  16. 2020Learning Discrete Distributions by Dequantizationcited by 18
  17. 2020The Convolution Exponential and Generalized Sylvester Flowscited by 18
  18. 2013High-Dimensional Probability Estimation with Deep Density Modelscited by 17

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 2; 2023: 6; 2024: 4; 2025: 3.

17 papers.

Sort:

By area:

  1. 2021Densely connected normalizing flowscited by 49
  2. 2023Learning Data Representations with Joint Diffusion Modelscited by 22
  3. 2023Estimating Treatment Effects from Irregular Time Series Observations with Hidden Confounderscited by 17
  4. 2022ButterflyFlow: Building Invertible Layers with Butterfly Matricescited by 12
  5. 2024Monotone, Bi-Lipschitz, and Polyak-Lojasiewicz Networkscited by 9
  6. 2022Invertible Monotone Operators for Normalizing Flowscited by 6
  7. 2023Lagrangian Flow Networks for Conservation Lawscited by 5
  8. 2024Entropy-Informed Weighting Channel Normalizing Flow for Deep Generative Modelscited by 5
  9. 2023Computing excited states of molecules using normalizing flowscited by 4
  10. 2023Estimating Treatment Effects in Continuous Time with Hidden Confounderscited by 3
  11. 2024LipKernel: Lipschitz-Bounded Convolutional Neural Networks via Dissipative Layerscited by 2
  12. 2025Deep Neural Networks Inspired by Differential Equationscited by 2
5 more
  1. 2023Discouraging posterior collapse in hierarchical Variational Autoencoders using contextcited by 1
  2. 2024Injective flows for star-like manifoldscited by 1
  3. 2025CDFlow: Building Invertible Layers with Circulant and Diagonal Matricescited by 1
  4. 2021Attentive Contractive Flow with Lipschitz-constrained Self-Attentioncited by 0
  5. 2025Posterior shape models revisited: Improving 3D reconstructions from partial data using target specific modelscited by 0