mlsanityData to 14 Oct 2026

Lifting Architectural Constraints of Injective Flows

arXiv 2306.01843 ↗First posted 2 Jun 2023cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

Normalizing Flows explicitly maximize a full-dimensional likelihood on the training data. However, real data is typically only supported on a lower-dimensional manifold leading the model to expend significant compute on modeling noise. Injective Flows fix this by jointly learning a manifold and the distribution on it. So far, they have been limited by restrictive architectures and/or high computational cost. We lift both constraints by a new efficient estimator for the maximum likelihood loss, compatible with free-form bottleneck architectures. We further show that naively learning both the data manifold and the distribution on it can lead to divergent solutions, and use this insight to motivate a stable maximum likelihood training objective. We perform extensive experiments on toy, tabular and image data, demonstrating the competitive performance of the resulting model.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 43 of its 65 references that are in the map. First posted on arXiv 2 Jun 2023.

Builds on

43 of 65 references.

References · 43

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2017Neural Discrete Representation Learningcited by 4,712
  5. 2013Auto-Encoding Variational Bayescited by 4,053
  6. 2018Neural Ordinary Differential Equationscited by 3,779
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  8. 2016Improved Techniques for Training GANscited by 2,970
  9. 2015Variational Inference with Normalizing Flowscited by 2,170
  10. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
  11. 2017Masked Autoregressive Flow for Density Estimationcited by 684
  12. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
31 more
  1. 2018Disentangling by Factorisingcited by 616
  2. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
  3. 2015Importance Weighted Autoencoderscited by 456
  4. 2017Wasserstein Auto-Encoderscited by 373
  5. 2018Invertible Residual Networkscited by 365
  6. 2015Adversarial Autoencoderscited by 364
  7. 2019Residual Flows for Invertible Generative Modelingcited by 257
  8. 2018Analyzing Inverse Problems with Invertible Neural Networkscited by 242
  9. 2020BayesFlow: Learning complex stochastic models with invertible neural networkscited by 144
  10. 2019From Variational to Deterministic Autoencoderscited by 133
  11. 2017InfoVAE: Information Maximizing Variational Autoencoderscited by 124
  12. 2020Flows for simultaneous manifold learning and density estimationcited by 118
  13. 2021Rectangular Flows for Manifold Learningcited by 50
  14. 2015Learning to Generate Images with Perceptual Similarity Metricscited by 39
  15. 2021Trumpets: Injective Flows for Inference and Inverse Problemscited by 36
  16. 2021Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flowscited by 31
  17. 2022Pythae: Unifying Generative Autoencoders in Python -- A Benchmarking Use Casecited by 26
  18. 2020Regularized Autoencoders via Relaxed Injective Probability Flowcited by 23
  19. 2020Normalizing Flows Across Dimensionscited by 19
  20. 2020Relative gradient optimization of the Jacobian term in unsupervised deep learningcited by 16
  21. 2019Perceptual Generative Autoencoderscited by 15
  22. 2020Probabilistic Autoencodercited by 13
  23. 2020Self Normalizing Flowscited by 12
  24. 2018Invertible Autoencoder for domain adaptationcited by 10
  25. 2022Nonlinear Isometric Manifold Learning for Injective Normalizing Flowscited by 6
  26. 2022Whitening Convergence Rate of Coupling-based Normalizing Flowscited by 6
  27. 2022Denoising Deep Generative Modelscited by 6
  28. 2021PIE: Pseudo-Invertible Encodercited by 5
  29. 2021Spread Flows for Manifold Modellingcited by 4
  30. 2022Deterministic training of generative autoencoders using invertible layerscited by 4
  31. 2020Batch norm with entropic regularization turns deterministic autoencoders into generative modelscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 6; 2025: 4; 2026: 4.

18 papers.

Sort:

By area:

  1. 2023Manifold Preserving Guided Diffusioncited by 157
  2. 2024Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connectionscited by 47
  3. 2023Free-form Flows: Make Any Architecture a Normalizing Flowcited by 26
  4. 2023A Review of Change of Variable Formulas for Generative Modelingcited by 13
  5. 2025FALCON: Few-step Accurate Likelihoods for Continuous Flowscited by 13
  6. 2024Fast constrained sampling in pre-trained diffusion modelscited by 10
  7. 2023Learning Distributions on Manifolds with Free-Form Flowscited by 7
  8. 2024Efficient Distribution Matching of Representations via Noise-Injected Deep InfoMaxcited by 4
  9. 2026Understanding Self-Supervised Learning via Latent Distribution Matchingcited by 4
  10. 2024TRADE: Transfer of Distributions between External Conditions with Normalizing Flowscited by 3
  11. 2025Learning Geometry and Topology via Multi-Chart Flowscited by 3
  12. 2025On Convolutions, Intrinsic Dimension, and Diffusion Modelscited by 3
6 more
  1. 2026CrossFlow: One-Step Generation Across Latent and Pixel Spacescited by 2
  2. 2024Optimizing Retinal Prosthetic Stimuli with Conditional Invertible Neural Networkscited by 1
  3. 2024Injective flows for star-like manifoldscited by 1
  4. 2025Beyond Diagonal Covariance: Flexible Posterior VAEs via Free-Form Injective Flowscited by 1
  5. 2026Show Me What You Don't Know: Efficient Sampling from Invariant Sets for Model Validationcited by 0
  6. 2026NAE: Normalizing AutoEncodercited by 0