mlsanityData to 7 Oct 2026

Invertible Monotone Operators for Normalizing Flows

arXiv 2210.08176 ↗First posted 15 Oct 2022cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Normalizing flows model probability distributions by learning invertible transformations that transfer a simple distribution into complex distributions. Since the architecture of ResNet-based normalizing flows is more flexible than that of coupling-based models, ResNet-based normalizing flows have been widely studied in recent years. Despite their architectural flexibility, it is well-known that the current ResNet-based models suffer from constrained Lipschitz constants. In this paper, we propose the monotone formulation to overcome the issue of the Lipschitz constants using monotone operators and provide an in-depth theoretical analysis. Furthermore, we construct an activation function called Concatenated Pila (CPila) to improve gradient flow. The resulting model, Monotone Flows, exhibits an excellent performance on multiple density estimation benchmarks (MNIST, CIFAR-10, ImageNet32, ImageNet64). Code is available at https://github.com/mlvlab/MonotoneFlows.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 39 of its 55 references that are in the map. First posted on arXiv 15 Oct 2022.

Builds on

39 of 55 references.

References · 39

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Variational Inference with Normalizing Flowscited by 2,161
  3. 2016Density estimation using Real NVPcited by 2,029
  4. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  5. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  6. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  7. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  8. 2016Pixel Recurrent Neural Networkscited by 814
  9. 2017Masked Autoregressive Flow for Density Estimationcited by 680
  10. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 670
  11. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  12. 2019Deep Equilibrium Modelscited by 603
27 more
  1. 2019Neural Spline Flowscited by 590
  2. 2018Invertible Residual Networkscited by 364
  3. 2021CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flowscited by 320
  4. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  5. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
  6. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
  7. 2019Residual Flows for Invertible Generative Modelingcited by 256
  8. 2020SRFlow: Learning the Super-Resolution Space with Normalizing Flowcited by 209
  9. 2018Neural Autoregressive Flowscited by 207
  10. 2019Flow-based generative models for Markov chain Monte Carlo in lattice field theorycited by 147
  11. 2021Fully Convolutional Cross-Scale-Flows for Image-based Defect Detectioncited by 117
  12. 2019Noise Flow: Noise Modeling with Conditional Normalizing Flowscited by 108
  13. 2020Monotone operator equilibrium networkscited by 100
  14. 2020Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimizationcited by 89
  15. 2020Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximatorscited by 85
  16. 2020Anomaly Detection with Density Estimationcited by 71
  17. 2021Invertible Denoising Network: A Light Solution for Real Noise Removalcited by 68
  18. 2020Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Modelscited by 66
  19. 2020Learning Maximally Monotone Operators for Image Recoverycited by 62
  20. 2021Smooth Normalizing Flowscited by 52
  21. 2021Densely connected normalizing flowscited by 49
  22. 2020VFlow: More Expressive Generative Flows with Variational Data Augmentationcited by 38
  23. 2021Implicit Normalizing Flowscited by 28
  24. 2020The Convolution Exponential and Generalized Sylvester Flowscited by 18
  25. 2021Invertible DenseNets with Concatenated LipSwishcited by 17
  26. 2020Woodbury Transformations for Deep Generative Flowscited by 10
  27. 2021Normalizing Flow as a Flexible Fidelity Objective for Photo-Realistic Super-resolutioncited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 0; 2026: 2.

6 papers.

  1. 2024Monotone, Bi-Lipschitz, and Polyak-Lojasiewicz Networkscited by 9
  2. 2024Learning truly monotone operators with applications to nonlinear inverse problemscited by 2
  3. 2024Learning Stable and Passive Neural Differential Equationscited by 2
  4. 2023MixerFlow: MLP-Mixer meets Normalising Flowscited by 0
  5. 2026A lift for input-convex neural net trainingcited by 0
  6. 2026Diagnosing Under-Development of Irreversible Processes in Video Generationcited by 0