mlsanityData to 7 Oct 2026

Intermediate Layer Optimization for Inverse Problems using Deep Generative Models

arXiv 2102.07364 ↗First posted 15 Feb 2021cs.LGcited by 70

Not on the map. Why?

Cited by 22 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose Intermediate Layer Optimization (ILO), a novel optimization algorithm for solving inverse problems with deep generative models. Instead of optimizing only over the initial latent code, we progressively change the input layer obtaining successively more expressive generators. To explore the higher dimensional spaces, our method searches for latent codes that lie within a small l1 ball around the manifold induced by the previous layer. Our theoretical analysis shows that by keeping the radius of the ball relatively small, we can improve the established error bound for compressed sensing with deep generative models. We empirically show that our approach outperforms state-of-the-art methods introduced in StyleGAN-2 and PULSE for a wide range of inverse problems including inpainting, denoising, super-resolution and compressed sensing.

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). Builds on 40 of its 74 references that are in the map. First posted on arXiv 15 Feb 2021.

Builds on

40 of 74 references.

References · 40

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  6. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  7. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  8. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  9. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  10. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  11. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  12. 2020PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Modelscited by 358
28 more
  1. 2017Compressed Sensing using Generative Modelscited by 337
  2. 2020Deep Learning Techniques for Inverse Problems in Imagingcited by 290
  3. 2018Generating Adversarial Examples with Adversarial Networkscited by 256
  4. 2018DeepFakes: a New Threat to Face Recognition? Assessment and Detectioncited by 208
  5. 2018Deep Learning for Single Image Super-Resolution: A Brief Reviewcited by 158
  6. 2019Seeing What a GAN Cannot Generatecited by 148
  7. 2019Deep Learning on Image Denoising: An overviewcited by 128
  8. 2019Deep Learning for Deepfakes Creation and Detection: A Surveycited by 102
  9. 2019Coherent Semantic Attention for Image Inpaintingcited by 97
  10. 2019Invertible generative models for inverse problems: mitigating representation error and dataset biascited by 93
  11. 2018Phase Retrieval Under a Generative Priorcited by 73
  12. 2017One Network to Solve Them All --- Solving Linear Inverse Problems using Deep Projection Modelscited by 67
  13. 2018Solving Linear Inverse Problems Using GAN Priors: An Algorithm with Provable Guaranteescited by 58
  14. 2019Learned Image Downscaling for Upscaling using Content Adaptive Resamplercited by 55
  15. 2019Deep Compressed Sensingcited by 46
  16. 2020Improving the Fairness of Deep Generative Models without Retrainingcited by 46
  17. 2017Global Guarantees for Enforcing Deep Generative Priors by Empirical Riskcited by 37
  18. 2018Modeling Sparse Deviations for Compressed Sensing using Generative Modelscited by 33
  19. 2019Block Coordinate Regularization by Denoisingcited by 33
  20. 2019Inverting Deep Generative models, One layer at a timecited by 33
  21. 2020Imperfect ImaGANation: Implications of GANs Exacerbating Biases on Facial Data Augmentation and Snapchat Selfie Lensescited by 27
  22. 2019Information-Theoretic Lower Bounds for Compressive Sensing with Generative Modelscited by 25
  23. 2019Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Modelscited by 21
  24. 2020Defending against GAN-based Deepfake Attacks via Transformation-aware Adversarial Facescited by 21
  25. 2018Task-Aware Compressed Sensing with Generative Adversarial Networkscited by 17
  26. 2018Correction by Projection: Denoising Images with Generative Adversarial Networkscited by 11
  27. 2019Surfing: Iterative optimization over incrementally trained deep networkscited by 5
  28. 2019Asymptotics of MAP Inference in Deep Networkscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 16; 2023: 15; 2024: 11; 2025: 8; 2026: 8.

70 papers.

Sort:

By area:

  1. 2022Denoising Diffusion Restoration Modelscited by 951
  2. 2022Improving Diffusion Models for Inverse Problems using Manifold Constraintscited by 501
  3. 2021Robust Compressed Sensing MRI with Deep Generative Priorscited by 284
  4. 2023DiffBIR: Towards Blind Image Restoration with Generative Diffusion Priorcited by 255
  5. 2024A Survey on Diffusion Models for Inverse Problemscited by 173
  6. 2021D2C: Diffusion-Denoising Models for Few-shot Conditional Generationcited by 96
  7. 2021FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimizationcited by 71
  8. 2023Training-free Linear Image Inverses via Flowscited by 70
  9. 2022Generative Joint Source-Channel Coding for Semantic Image Transmissioncited by 55
  10. 2021Labels4Free: Unsupervised Segmentation using StyleGANcited by 53
  11. 2022MIMO Channel Estimation using Score-Based Generative Modelscited by 50
  12. 2023GIFD: A Generative Gradient Inversion Method with Feature Domain Optimizationcited by 45
38 more
  1. 2022Learning to Solve PDE-constrained Inverse Problems with Graph Networkscited by 37
  2. 2022Multiresolution Textual Inversioncited by 33
  3. 2024Holo-Relighting: Controllable Volumetric Portrait Relighting from a Single Imagecited by 32
  4. 2021Generative Adversarial Registration for Improved Conditional Deformable Templatescited by 31
  5. 2024Blind Image Restoration via Fast Diffusion Inversioncited by 29
  6. 2024Idempotence and Perceptual Image Compressioncited by 27
  7. 2021Regularising Inverse Problems with Generative Machine Learning Modelscited by 22
  8. 2022Theoretical Perspectives on Deep Learning Methods in Inverse Problemscited by 22
  9. 2021Prior Image-Constrained Reconstruction using Style-Based Generative Modelscited by 19
  10. 2024A Closer Look at GAN Priors: Exploiting Intermediate Features for Enhanced Model Inversion Attackscited by 17
  11. 2021Designing Counterfactual Generators using Deep Model Inversioncited by 15
  12. 2024Repulsive Latent Score Distillation for Solving Inverse Problemscited by 15
  13. 2023Robust Unsupervised StyleGAN Image Restorationcited by 14
  14. 2023CommIN: Semantic Image Communications as an Inverse Problem with INN-Guided Diffusion Modelscited by 14
  15. 2025LATINO-PRO: LAtent consisTency INverse sOlver with PRompt Optimizationcited by 14
  16. 2021EdiBERT, a generative model for image editingcited by 13
  17. 2022TurbuGAN: An Adversarial Learning Approach to Spatially-Varying Multiframe Blind Deconvolution with Applications to Imaging Through Turbulencecited by 11
  18. 2022Score-Guided Intermediate Layer Optimization: Fast Langevin Mixing for Inverse Problemscited by 11
  19. 2024OmniSSR: Zero-shot Omnidirectional Image Super-Resolution using Stable Diffusion Modelcited by 11
  20. 2023Deep Internal Learning: Deep Learning from a Single Inputcited by 9
  21. 2024Consistency Model is an Effective Posterior Sample Approximation for Diffusion Inverse Solverscited by 8
  22. 2021Robust 1-bit Compressive Sensing with Partial Gaussian Circulant Matrices and Generative Priorscited by 7
  23. 2021Solving Inverse Problems with NerfGANscited by 7
  24. 2023A Unified Framework for Uniform Signal Recovery in Nonlinear Generative Compressed Sensingcited by 7
  25. 2025Integrating Intermediate Layer Optimization and Projected Gradient Descent for Solving Inverse Problems with Diffusion Modelscited by 7
  26. 2023Inverse problem regularization with hierarchical variational autoencoderscited by 6
  27. 2024DiffBody: Human Body Restoration by Imagining with Generative Diffusion Priorcited by 6
  28. 2025CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Samplingcited by 5
  29. 2022Overparameterization Improves StyleGAN Inversioncited by 4
  30. 2023Unsupervised approaches based on optimal transport and convex analysis for inverse problems in imagingcited by 4
  31. 2023Variational Bayes image restoration with compressive autoencoderscited by 4
  32. 2025Iso-Riemannian Optimization on Learned Data Manifoldscited by 4
  33. 2026AI4SLT: Empirical Processes in Lean 4 for Formal Statistical Learning Theorycited by 4
  34. 2021Differentiable Gaussianization Layers for Inverse Problems Regularized by Deep Generative Modelscited by 3
  35. 2022Equivariant Priors for Compressed Sensing with Unknown Orientationcited by 3
  36. 2023High-Dimensional MR Reconstruction Integrating Subspace and Adaptive Generative Modelscited by 3
  37. 2022Accelerating Inverse Learning via Intelligent Localization with Exploratory Samplingcited by 2
  38. 2025Learning Single Index Models with Diffusion Priorscited by 2

50 most cited of 70 All 70 (JSON)