mlsanityData to 14 Oct 2026

SILO: Solving Inverse Problems with Latent Operators

arXiv 2501.11746 ↗First posted 20 Jan 2025cs.CV cs.AI cs.LGcited by 19

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

Consistent improvement of image priors over the years has led to the development of better inverse problem solvers. Diffusion models are the newcomers to this arena, posing the strongest known prior to date. Recently, such models operating in a latent space have become increasingly predominant due to their efficiency. In recent works, these models have been applied to solve inverse problems. Working in the latent space typically requires multiple applications of an Autoencoder during the restoration process, which leads to both computational and restoration quality challenges. In this work, we propose a new approach for handling inverse problems with latent diffusion models, where a learned degradation function operates within the latent space, emulating a known image space degradation. Usage of the learned operator reduces the dependency on the Autoencoder to only the initial and final steps of the restoration process, facilitating faster sampling and superior restoration quality. We demonstrate the effectiveness of our method on a variety of image restoration tasks and datasets, achieving significant improvements over prior art.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 37 of its 59 references that are in the map. First posted on arXiv 20 Jan 2025.

Builds on

37 of 59 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusion1 Dec
  2. 2023Regularization by Texts for Latent Diffusion Inverse Solvers27 Nov
  3. 2023Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency16 Jul
  4. 2023Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models2 Jul
  5. 2023Denoising Diffusion Models for Plug-and-Play Image Restoration15 May
  6. 2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model1 Dec
  7. 2022Diffusion Posterior Sampling for General Noisy Inverse Problems29 Sep
  8. 2022Denoising Diffusion Restoration Models27 Jan

Other references · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  11. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  12. 2022Classifier-Free Diffusion Guidancecited by 5,847
17 more
  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  3. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,887
  4. 2018Demystifying MMD GANscited by 1,293
  5. 2017Deep Image Priorcited by 1,098
  6. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 970
  7. 2017The Perception-Distortion Tradeoffcited by 587
  8. 2020Plug-and-Play Image Restoration with Deep Denoiser Priorcited by 470
  9. 2016The Little Engine that Could: Regularization by Denoising (RED)cited by 307
  10. 2015Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restorationcited by 195
  11. 2023Image Denoising: The Deep Learning Revolution and Beyond -- A Survey Paper --cited by 71
  12. 2021A Theory of the Distortion-Perception Tradeoff in Wasserstein Spacecited by 52
  13. 2023Readout Guidance: Learning Control from Diffusion Featurescited by 52
  14. 2024DEFT: Efficient Fine-Tuning of Diffusion Models by Learning the Generalised h-transformcited by 46
  15. 2021High Perceptual Quality Image Denoising with a Posterior Sampling CGANcited by 33
  16. 2024Adaptive Compressed Sensing with Diffusion-Based Posterior Samplingcited by 13
  17. 2022High-Perceptual Quality JPEG Decoding via Posterior Samplingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 10.

19 papers.

Sort:

By area:

  1. 2025Compressed Image Generation with Denoising Diffusion Codebook Modelscited by 28
  2. 2025Appa: Bending Weather Dynamics with Latent Diffusion Models for Global Data Assimilationcited by 20
  3. 2025Diffusion models for inverse problemscited by 16
  4. 2025Turbo-DDCM: Fast and Flexible Zero-Shot Diffusion-Based Image Compressioncited by 8
  5. 2025Inverse Problem Sampling in Latent Space Using Sequential Monte Carlocited by 7
  6. 2025InverseCrafter: Efficient Video ReCapture as a Latent Domain Inverse Problemcited by 6
  7. 2026Consistency Regularised Gradient Flows for Inverse Problemscited by 2
  8. 2026Image Restoration via Diffusion Models with Dynamic Resolutioncited by 2
  9. 2026Measurement-Consistent Langevin Corrector for Stabilizing Latent Diffusion Inverse Problem Solverscited by 1
  10. 2026SP3: Spherical Priors for Plug-and-Play Restorationcited by 1
  11. 2025LatentINDIGO: An INN-Guided Latent Diffusion Algorithm for Image Restorationcited by 0
  12. 2025MACS: Measurement-Aware Consistency Sampling for Inverse Problemscited by 0
7 more
  1. 2025Align & Invert: Solving Inverse Problems with Diffusion and Flow-based Models via Representation Alignmentcited by 0
  2. 2026Posterior Continuation with Noise-Conditioned Frequency Exposure for Diffusion Inverse Problemscited by 0
  3. 2026Accelerating Video Inverse Problem Solvers with Autoregressive Diffusion Modelscited by 0
  4. 2026Music Restoration via Latent Operator Optimization and Diffusion Model Priorscited by 0
  5. 2026ANaLOG: Anisotropic Native-Latent Operator Guidance for Solving Inverse Problemscited by 0
  6. 2026LoCoVSR: Local Context Diffusion Posterior Sampling for Video Super-Resolutioncited by 0
  7. 2026BAM! Bayesian Anything Model: a foundation model for generative computational imagingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Perturb-and-Solve: Efficient Learned-Operator Conditioning for Latent Diffusion Inverse Problems
  2. 2024A Survey on Diffusion Models for Inverse Problems
  3. 2023Prompt-tuning latent diffusion models for inverse problems
  4. 2023Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion Models
  5. 2024Blind Inversion using Latent Diffusion Priors
  6. 2024Blind Inverse Problem Solving Made Easy by Text-to-Image Latent Diffusion
  7. 2024Prototype Clustered Diffusion Models for Versatile Inverse Problems
  8. 2025SAIP: A Plug-and-Play Scale-adaptive Module in Diffusion-based Inverse Problems