mlsanityData to 7 Oct 2026

Universal approximation property of invertible neural networks

arXiv 2204.07415 ↗First posted 15 Apr 2022cs.LG cs.NE stat.MLcited by 30

Not on the map. Why?

Abstract (arXiv)

Invertible neural networks (INNs) are neural network architectures with invertibility by design. Thanks to their invertibility and the tractability of Jacobian, INNs have various machine learning applications such as probabilistic modeling, generative modeling, and representation learning. However, their attractive properties often come at the cost of restricting the layer designs, which poses a question on their representation power: can we use these models to approximate sufficiently diverse functions? To answer this question, we have developed a general theoretical framework to investigate the representation power of INNs, building on a structure theorem of differential geometry. The framework simplifies the approximation problem of diffeomorphisms, which enables us to show the universal approximation properties of INNs. We apply the framework to two representative classes of INNs, namely Coupling-Flow-based INNs (CF-INNs) and Neural Ordinary Differential Equations (NODEs), and elucidate their high representation power despite the restrictions on their architectures.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 39 of its 74 references that are in the map. First posted on arXiv 15 Apr 2022.

Builds on

39 of 74 references.

References · 39

  1. 2018Neural Ordinary Differential Equationscited by 3,733
  2. 2016Density estimation using Real NVPcited by 2,029
  3. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  4. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  5. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  6. 2017Masked Autoregressive Flow for Density Estimationcited by 680
  7. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  8. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  9. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
  10. 2018Analyzing Inverse Problems with Invertible Neural Networkscited by 239
  11. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  12. 2018Sorting out Lipschitz function approximationcited by 211
27 more
  1. 2018Neural Autoregressive Flowscited by 207
  2. 2017Multiplicative Normalizing Flows for Variational Bayesian Neural Networkscited by 157
  3. 2019Flow-based generative models for Markov chain Monte Carlo in lattice field theorycited by 147
  4. 2020SoftFlow: Probabilistic Framework for Normalizing Flow on Manifoldscited by 97
  5. 2020Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimizationcited by 89
  6. 2020Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximatorscited by 85
  7. 2019Deep Learning via Dynamical Systems: An Approximation Perspectivecited by 78
  8. 2019Semi-Supervised Learning with Normalizing Flowscited by 76
  9. 2019Latent Normalizing Flows for Discrete Sequencescited by 66
  10. 2019Block Neural Autoregressive Flowcited by 62
  11. 2019Sum-of-Squares Polynomial Flowcited by 62
  12. 2018Resampled Priors for Variational Autoencoderscited by 56
  13. 2021Neural ODE control for classification, approximation and transportcited by 56
  14. 2018FloWaveNet : A Generative Flow for Raw Audiocited by 54
  15. 2020Few-shot Domain Adaptation by Causal Mechanism Transfercited by 47
  16. 2019Hybrid Models with Deep and Invertible Featurescited by 41
  17. 2019Improving Exploration in Soft-Actor-Critic with Normalizing Flows Policiescited by 37
  18. 2020Universal Approximation Property of Neural Ordinary Differential Equationscited by 32
  19. 2021Universal Joint Approximation of Manifolds and Densities by Simple Injective Flowscited by 13
  20. 2019Density Matching for Bilingual Word Embeddingcited by 11
  21. 2021Learning the Koopman Eigendecomposition: A Diffeomorphic Approachcited by 9
  22. 2021Universal Approximation of Residual Flows in Maximum Mean Discrepancycited by 7
  23. 2020ChartPointFlow for Topology-Aware 3D Point Cloud Generationcited by 6
  24. 2021Abelian Neural Networkscited by 2
  25. 2022Universality of parametric Coupling Flows over parametric diffeomorphismscited by 2
  26. 2021Minimax Analysis for Inverse Risk in Nonparametric Planer Invertible Regressioncited by 1
  27. 2021ELF: Exact-Lipschitz Based Universal Density Approximator Flowcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 7; 2024: 8; 2025: 7; 2026: 5.

30 papers.

Sort:

By area:

  1. 2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
  2. 2022Identifiability of deep generative models without auxiliary informationcited by 70
  3. 2022Reliable amortized variational inference with physics-based latent distribution correctioncited by 25
  4. 2023Distribution learning via neural differential equations: a nonparametric statistical perspectivecited by 22
  5. 2024On the Universality of Volume-Preserving and Coupling-Based Normalizing Flowscited by 18
  6. 2023Globally injective and bijective neural operatorscited by 13
  7. 2022Whitening Convergence Rate of Coupling-based Normalizing Flowscited by 6
  8. 2024Enhancing Accuracy in Generative Models via Knowledge Transfercited by 6
  9. 2025Efficient Regression-Based Training of Normalizing Flows for Boltzmann Generatorscited by 6
  10. 2024Stable Training of Normalizing Flows for High-dimensional Variational Inferencecited by 5
  11. 2023Cluster-based classification with neural ODEs via controlcited by 4
  12. 2024Diffeomorphic Measure Matching with Kernels for Generative Modelingcited by 4
18 more
  1. 2024Adaptive deep density approximation for stochastic dynamical systemscited by 4
  2. 2025Numerical and statistical analysis of NeuralODE with Runge-Kutta time integrationcited by 3
  3. 2025Generative AI for Autonomous Driving: A Reviewcited by 3
  4. 2023Enriching Disentanglement: From Logical Definitions to Quantitative Metricscited by 2
  5. 2023On the Approximation of Bi-Lipschitz Maps by Invertible Neural Networkscited by 2
  6. 2024Can neural operators always be continuously discretized?cited by 2
  7. 2024Deep Neural Networks with Symplectic Preservation Propertiescited by 1
  8. 2025An invertible generative model for forward and inverse problemscited by 1
  9. 2025Autonomous-Flow-Based Generationcited by 1
  10. 2026Exact Fixed-Point Constraints in Neural-ODEs with Provable Universalitycited by 1
  11. 2023Using Autoencoders and AutoDiff to Reconstruct Missing Variables in a Set of Time Seriescited by 0
  12. 2024GradINN: Gradient Informed Neural Networkcited by 0
  13. 2025RIE-SenseNet: Riemannian Manifold Embedding of Multi-Source Industrial Sensor Signals for Robust Pattern Recognitioncited by 0
  14. 2025Learning Geometrically-Informed Lyapunov Functions with Deep Diffeomorphic RBF Networkscited by 0
  15. 2026Sampling Boltzmann distributions via normalizing flow approximation of transport mapscited by 0
  16. 2026Expressivity of Bi-Lipschitz Normalizing Flows: A Score-Based Diffusion Perspectivecited by 0
  17. 2026Approximating velocity fields with planted attractors via Neural-ODEs for classification purposescited by 0
  18. 2026Kinks vs. Smoothness: Identifiability of Real Analytic nICA for Laplace-like Sourcescited by 0