mlsanityData to 14 Oct 2026

LATINO-PRO: LAtent consisTency INverse sOlver with PRompt Optimization

arXiv 2503.12615 ↗First posted 16 Mar 2025cs.CV cs.LGcited by 16

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

Text-to-image latent diffusion models (LDMs) have recently emerged as powerful generative models with great potential for solving inverse problems in imaging. However, leveraging such models in a Plug & Play (PnP), zero-shot manner remains challenging because it requires identifying a suitable text prompt for the unknown image of interest. Also, existing text-to-image PnP approaches are highly computationally expensive. We herein address these challenges by proposing a novel PnP inference paradigm specifically designed for embedding generative models within stochastic inverse solvers, with special attention to Latent Consistency Models (LCMs), which distill LDMs into fast generators. We leverage our framework to propose LAtent consisTency INverse sOlver (LATINO), the first zero-shot PnP framework to solve inverse problems with priors encoded by LCMs. Our conditioning mechanism avoids automatic differentiation and reaches SOTA quality in as little as 8 neural function evaluations. As a result, LATINO delivers remarkably accurate solutions and is significantly more memory and computationally efficient than previous approaches. We then embed LATINO within an empirical Bayesian framework that automatically calibrates the text prompt from the observed measurements by marginal maximum likelihood estimation. Extensive experiments show that prompt self-calibration greatly improves estimation, allowing LATINO with PRompt Optimization to define new SOTAs in image reconstruction quality and computational efficiency. The code is available at https://latino-pro.github.io

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 53 of its 68 references that are in the map. First posted on arXiv 16 Mar 2025.

Builds on

53 of 68 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024Zero-Shot Image Restoration Using Few-Step Guidance of Consistency Models (and Beyond)29 Dec
  2. 2024Variational Diffusion Posterior Sampling with Midpoint Guidance13 Oct
  3. 2024Empirical Bayesian image restoration by Langevin sampling with a denoising diffusion implicit prior6 Sep
  4. 2024CoSIGN: Few-Step Guidance of ConSIstency Model to Solve General INverse Problems17 Jul
  5. 2024Consistency Model is an Effective Posterior Sample Approximation for Diffusion Inverse Solvers9 Feb
  6. 2023Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusion1 Dec
  7. 2023Regularization by Texts for Latent Diffusion Inverse Solvers27 Nov
  8. 2023Prompt-tuning latent diffusion models for inverse problems2 Oct
  9. 2023Fast and Stable Diffusion Inverse Solver with History Gradient Update22 Jul
  10. 2023Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency16 Jul
  11. 2023Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models2 Jul
  12. 2023Denoising Diffusion Models for Plug-and-Play Image Restoration15 May
2 more
  1. 2022Diffusion Posterior Sampling for General Noisy Inverse Problems29 Sep
  2. 2022Denoising Diffusion Restoration Models27 Jan

Other references · 39

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  9. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  10. 2022Classifier-Free Diffusion Guidancecited by 5,847
  11. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
27 more
  1. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  2. 2023Consistency Modelscited by 1,822
  3. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  4. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  5. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 846
  6. 2022On Distillation of Guided Diffusion Modelscited by 725
  7. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 684
  8. 2020Plug-and-Play Image Restoration with Deep Denoiser Priorcited by 470
  9. 2020PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Modelscited by 358
  10. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speedcited by 348
  11. 2017Compressed Sensing using Generative Modelscited by 339
  12. 2016The Little Engine that Could: Regularization by Denoising (RED)cited by 307
  13. 2020Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulationcited by 225
  14. 2023LCM-LoRA: A Universal Stable-Diffusion Acceleration Modulecited by 211
  15. 2015Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithmcited by 197
  16. 2017Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problemscited by 107
  17. 2021Bayesian imaging using Plug & Play priors: when Langevin meets Tweediecited by 97
  18. 2021Intermediate Layer Optimization for Inverse Problems using Deep Generative Modelscited by 70
  19. 2018Split-and-augmented Gibbs sampler - Application to large-scale inference problemscited by 47
  20. 2013Proximal Markov chain Monte Carlo algorithmscited by 46
  21. 2023Plug-and-Play split Gibbs sampler: embedding deep generative priors in Bayesian inferencecited by 42
  22. 2021Solving Inverse Problems by Joint Posterior Maximization with Autoencoding Priorcited by 24
  23. 2021Bayesian Imaging With Data-Driven Priors Encoded by Neural Networks: Theory, Methods, and Algorithmscited by 24
  24. 2024Plug-and-Play image restoration with Stochastic deNOising REgularizationcited by 17
  25. 2022The split Gibbs sampler revisited: improvements to its algorithmic structure and augmented target distributioncited by 11
  26. 2023Normalizing flow sampling with Langevin dynamics in the latent spacecited by 7
  27. 2023Inverse problem regularization with hierarchical variational autoencoderscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 6.

16 papers.

Sort:

By area:

  1. 2026Diamond Maps: Efficient Reward Alignment via Stochastic Flow Mapscited by 29
  2. 2025GLASS Flows: Transition Sampling for Alignment of Flow and Diffusion Modelscited by 17
  3. 2025Learning few-step posterior samplers by unfolding and distillation of diffusion modelscited by 6
  4. 2025InverseCrafter: Efficient Video ReCapture as a Latent Domain Inverse Problemcited by 6
  5. 2025Efficient Zero-Shot Inpainting with Decoupled Diffusion Guidancecited by 5
  6. 2025Normalization-equivariant Diffusion Models: Learning Posterior Samplers From Noisy And Partial Measurementscited by 3
  7. 2025LVTINO: LAtent Video consisTency INverse sOlver for High Definition Video Restorationcited by 2
  8. 2026Consistency Regularised Gradient Flows for Inverse Problemscited by 2
  9. 2026Fast and Robust Likelihood-Guided Diffusion Posterior Sampling with Amortized Variational Inferencecited by 1
  10. 2025Hypothesis Testing in Imaging Inverse Problemscited by 0
  11. 2025A Gradient Flow Approach to Solving Inverse Problems with Latent Diffusion Modelscited by 0
  12. 2025MACS: Measurement-Aware Consistency Sampling for Inverse Problemscited by 0
4 more
  1. 2025Bayesian model selection and misspecification testing in imaging inverse problems only from noisy and partial measurementscited by 0
  2. 2026Deep unfolding of MCMC kernels: scalable, modular & explainable GANs for high-dimensional posterior samplingcited by 0
  3. 2026BAM! Bayesian Anything Model: a foundation model for generative computational imagingcited by 0
  4. 2026Iterating Consistency Models: Stability, Error Bounds and Noise Schedulescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025PnP-CM: Consistency Models as Plug-and-Play Priors for Inverse Problems
  2. 2025Solving Inverse Problems with FLAIR
  3. 2026Stochastic Generative Plug-and-Play Priors
  4. 2026Latent Bias Alignment for High-Fidelity Diffusion Inversion in Real-World Image Reconstruction and Manipulation
  5. 2024A Survey on Diffusion Models for Inverse Problems
  6. 2025SILO: Solving Inverse Problems with Latent Operators
  7. 2025Inverse Problem Sampling in Latent Space Using Sequential Monte Carlo
  8. 2023Inversion-Free Image Editing with Natural Language