# LU-Net: Invertible Neural Networks Based on Matrix Factorization

arXiv: https://arxiv.org/abs/2302.10524 · HTML: https://mlsanity.com/paper/2302.10524/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 38 references that are in the map.
- First posted on arXiv 21 Feb 2023.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [A note on the evaluation of generative models](https://mlsanity.com/paper/1511.01844/) (2015)
- [Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design](https://mlsanity.com/paper/1902.00275/) (2019)
- [Analyzing Inverse Problems with Invertible Neural Networks](https://mlsanity.com/paper/1808.04730/) (2018)
- [Universal Approximation with Deep Narrow Networks](https://mlsanity.com/paper/1905.08539/) (2019)
- [Minimum Width for Universal Approximation](https://mlsanity.com/paper/2006.08859/) (2020)
- [Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators](https://mlsanity.com/paper/2006.11469/) (2020)
- [RNADE: The real-valued neural autoregressive density-estimator](https://mlsanity.com/paper/1306.0186/) (2013)
- [Densely connected normalizing flows](https://mlsanity.com/paper/2106.04627/) (2021)

## Built on by (most cited first, 8 of 8)

- [Learning Brenier Potentials with Convex Generative Adversarial Neural Networks](https://mlsanity.com/paper/2504.19779/) (2025)
- [How well do generative models solve inverse problems? A benchmark study](https://mlsanity.com/paper/2601.23238/) (2026)
- [VeriFlow: Modeling Distributions for Neural Network Verification](https://mlsanity.com/paper/2406.14265/) (2024)
- [Generative Design of Ship Propellers using Conditional Flow Matching](https://mlsanity.com/paper/2601.21637/) (2026)
- [Generative Inverse Design with Abstention via Diagonal Flow Matching](https://mlsanity.com/paper/2603.15925/) (2026)
- [Consistency of Learned Sparse Grid Quadrature Rules using NeuralODEs](https://mlsanity.com/paper/2507.01533/) (2025)
- [On Uniformly Scaling Flows: A Density-Aligned Approach to Deep One-Class Classification](https://mlsanity.com/paper/2510.09452/) (2025)
- [Generative Design of a Gas Turbine Combustor Using Invertible Neural Networks](https://mlsanity.com/paper/2604.24322/) (2026)

## Abstract (arXiv)

LU-Net is a simple and fast architecture for invertible neural networks (INN) that is based on the factorization of quadratic weight matrices mathsfA=LU, where mathsfL is a lower triangular matrix with ones on the diagonal and mathsfU an upper triangular matrix. Instead of learning a fully occupied matrix mathsfA, we learn mathsfL and mathsfU separately. If combined with an invertible activation function, such layers can easily be inverted whenever the diagonal entries of mathsfU are different from zero. Also, the computation of the determinant of the Jacobian matrix of such layers is cheap. Consequently, the LU architecture allows for cheap computation of the likelihood via the change of variables formula and can be trained according to the maximum likelihood principle. In our numerical experiments, we test the LU-net architecture as generative model on several academic datasets. We also provide a detailed comparison with conventional invertible neural networks in terms of performance, training as well as run time.