# Idempotent Generative Network

arXiv: https://arxiv.org/abs/2311.01462 · HTML: https://mlsanity.com/paper/2311.01462/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 19 references that are in the map.
- First posted on arXiv 2 Nov 2023.

## Builds on (most cited first, 14 of 14)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis](https://mlsanity.com/paper/1903.05628/) (2019)
- [Bayesian Flow Networks](https://mlsanity.com/paper/2308.07037/) (2023)
- [Combating Mode Collapse in GAN training: An Empirical Analysis using Hessian Eigenvalues](https://mlsanity.com/paper/2012.09673/) (2020)

## Built on by (most cited first, 18 of 18)

- [Diffusion-DICE: In-Sample Diffusion Guidance for Offline Reinforcement Learning](https://mlsanity.com/paper/2407.20109/) (2024)
- [Rethinking Score Distillation as a Bridge Between Image Distributions](https://mlsanity.com/paper/2406.09417/) (2024)
- [A Unified View of Score-Based and Drifting Models](https://mlsanity.com/paper/2603.07514/) (2026)
- [IT3: Idempotent Test-Time Training](https://mlsanity.com/paper/2410.04201/) (2024)
- [Energy-Based Flow Matching for Generating 3D Molecular Structure](https://mlsanity.com/paper/2508.18949/) (2025)
- [Discrete Distribution Networks](https://mlsanity.com/paper/2401.00036/) (2023)
- [Score-based Idempotent Distillation of Diffusion Models](https://mlsanity.com/paper/2509.21470/) (2025)
- [Who Said Neural Networks Aren't Linear?](https://mlsanity.com/paper/2510.08570/) (2025)
- [Conditional Idempotent Generative Networks](https://mlsanity.com/paper/2406.02841/) (2024)
- [Downscaling Neural Network for Coastal Simulations](https://mlsanity.com/paper/2408.16553/) (2024)
- [IWN: Image Watermarking Based on Idempotency](https://mlsanity.com/paper/2409.19506/) (2024)
- [Analytical Discovery of Manifold with Machine Learning](https://mlsanity.com/paper/2504.02511/) (2025)
- [A Non-Adversarial Approach to Idempotent Generative Modelling](https://mlsanity.com/paper/2511.02614/) (2025)
- [Context-dependent manifold learning: A neuromodulated constrained autoencoder approach](https://mlsanity.com/paper/2603.11673/) (2026)
- [RIVET: Robust Idempotent Voice Attribute Editing](https://mlsanity.com/paper/2606.19629/) (2026)
- [Encoder-Decoder Manifold Alignment for Idempotent Generation](https://mlsanity.com/paper/2606.22304/) (2026)
- [Moore, Escher, Penrose: A Conformal Golden Braid](https://mlsanity.com/paper/2610.02210/) (2026)
- [Correction Training for Diffusion Models](https://mlsanity.com/paper/2610.14136/) (2026)

## Abstract (arXiv)

We propose a new approach for generative modeling based on training a neural network to be idempotent. An idempotent operator is one that can be applied sequentially without changing the result beyond the initial application, namely f(f(z))=f(z). The proposed model f is trained to map a source distribution (e.g, Gaussian noise) to a target distribution (e.g. realistic images) using the following objectives: (1) Instances from the target distribution should map to themselves, namely f(x)=x. We define the target manifold as the set of all instances that f maps to themselves. (2) Instances that form the source distribution should map onto the defined target manifold. This is achieved by optimizing the idempotence term, f(f(z))=f(z) which encourages the range of f(z) to be on the target manifold. Under ideal assumptions such a process provably converges to the target distribution. This strategy results in a model capable of generating an output in one step, maintaining a consistent latent space, while also allowing sequential applications for refinement. Additionally, we find that by processing inputs from both target and source distributions, the model adeptly projects corrupted or modified data back to the target manifold. This work is a first step towards a ``global projector'' that enables projecting any input into a target data distribution.