# Invertible Residual Networks

arXiv: https://arxiv.org/abs/1811.00995 · HTML: https://mlsanity.com/paper/1811.00995/

## Key facts

- Cited by 365 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2018.

## Built on by (most cited first, 30 of 365)

- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion](https://mlsanity.com/paper/2211.14461/) (2022)
- [Sinkformers: Transformers with Doubly Stochastic Attention](https://mlsanity.com/paper/2110.11773/) (2021)
- [Score-Based Diffusion Models as Principled Priors for Inverse Imaging](https://mlsanity.com/paper/2304.11751/) (2023)
- [End-to-End Diffusion Latent Optimization Improves Classifier Guidance](https://mlsanity.com/paper/2303.13703/) (2023)
- [Deep Generative Learning via Schr"odinger Bridge](https://mlsanity.com/paper/2106.10410/) (2021)
- [Neural Flows: Efficient Alternative to Neural ODEs](https://mlsanity.com/paper/2110.13040/) (2021)
- [On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty](https://mlsanity.com/paper/2102.11409/) (2021)
- [On Training Implicit Models](https://mlsanity.com/paper/2111.05177/) (2021)
- [Efficient and Accurate Gradients for Neural SDEs](https://mlsanity.com/paper/2105.13493/) (2021)
- [Invertible Denoising Network: A Light Solution for Real Noise Removal](https://mlsanity.com/paper/2104.10546/) (2021)
- [Verifying the Union of Manifolds Hypothesis for Image Data](https://mlsanity.com/paper/2207.02862/) (2022)
- [Continual Learning via Local Module Composition](https://mlsanity.com/paper/2111.07736/) (2021)
- [On the Practicality of Deterministic Epistemic Uncertainty](https://mlsanity.com/paper/2107.00649/) (2021)
- [Towards Blind Watermarking: Combining Invertible and Non-invertible Mechanisms](https://mlsanity.com/paper/2212.12678/) (2022)
- [A Dynamical System Perspective for Lipschitz Neural Networks](https://mlsanity.com/paper/2110.12690/) (2021)
- [Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks](https://mlsanity.com/paper/2201.12179/) (2022)
- [CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphism](https://mlsanity.com/paper/2203.16529/) (2022)
- [Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Camera](https://mlsanity.com/paper/2206.15258/) (2022)
- [Rectangular Flows for Manifold Learning](https://mlsanity.com/paper/2106.01413/) (2021)
- [Score-based Generative Modeling Secretly Minimizes the Wasserstein Distance](https://mlsanity.com/paper/2212.06359/) (2022)
- [Densely connected normalizing flows](https://mlsanity.com/paper/2106.04627/) (2021)
- [A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness](https://mlsanity.com/paper/2205.00403/) (2022)
- [How Smooth Is Attention?](https://mlsanity.com/paper/2312.14820/) (2023)
- [MALI: A memory efficient and reverse accurate integrator for Neural ODEs](https://mlsanity.com/paper/2102.04668/) (2021)
- [Resampling Base Distributions of Normalizing Flows](https://mlsanity.com/paper/2110.15828/) (2021)
- [Reachability Analysis of Neural Feedback Loops](https://mlsanity.com/paper/2108.04140/) (2021)
- [Normalizing flow neural networks by JKO scheme](https://mlsanity.com/paper/2212.14424/) (2022)
- [Momentum Residual Neural Networks](https://mlsanity.com/paper/2102.07870/) (2021)

## Abstract (arXiv)

We show that standard ResNet architectures can be made invertible, allowing the same model to be used for classification, density estimation, and generation. Typically, enforcing invertibility requires partitioning dimensions or restricting network architectures. In contrast, our approach only requires adding a simple normalization step during training, already available in standard frameworks. Invertible ResNets define a generative model which can be trained by maximum likelihood on unlabeled data. To compute likelihoods, we introduce a tractable approximation to the Jacobian log-determinant of a residual block. Our empirical evaluation shows that invertible ResNets perform competitively with both state-of-the-art image classifiers and flow-based generative models, something that has not been previously achieved with a single architecture.