mlsanityData to 14 Oct 2026

Regularization by Texts for Latent Diffusion Inverse Solvers

arXiv 2311.15658 ↗First posted 27 Nov 2023cs.CV cs.AI cs.LGcited by 32

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

The recent development of diffusion models has led to significant progress in solving inverse problems by leveraging these models as powerful generative priors. However, challenges persist due to the ill-posed nature of such problems, often arising from ambiguities in measurements or intrinsic system symmetries. To address this, here we introduce a novel latent diffusion inverse solver, regularization by text (TReg), inspired by the human ability to resolve visual ambiguities through perceptual biases. TReg integrates textual descriptions of preconceptions about the solution during reverse diffusion sampling, dynamically reinforcing these descriptions through null-text optimization, which we refer to as adaptive negation. Our comprehensive experimental results demonstrate that TReg effectively mitigates ambiguity in inverse problems, improving both accuracy and efficiency.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 34 of its 43 references that are in the map. First posted on arXiv 27 Nov 2023.

Builds on

34 of 43 references.

Closest ancestors · 11

Same direction, newest first

  1. 2023Prompt-tuning latent diffusion models for inverse problems2 Oct
  2. 2023Fast and Stable Diffusion Inverse Solver with History Gradient Update22 Jul
  3. 2023Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency16 Jul
  4. 2023Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models2 Jul
  5. 2023A Variational Perspective on Solving Inverse Problems with Diffusion Models7 May
  6. 2023Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problems10 Mar
  7. 2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model1 Dec
  8. 2022Parallel Diffusion Models of Operator and Image for Blind Inverse Problems19 Nov
  9. 2022Diffusion Posterior Sampling for General Noisy Inverse Problems29 Sep
  10. 2022Denoising Diffusion Restoration Models27 Jan
  11. 2021Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction9 Dec

Other references · 23

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  9. 2022Classifier-Free Diffusion Guidancecited by 5,847
  10. 2013Auto-Encoding Variational Bayescited by 4,053
  11. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  12. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
11 more
  1. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
  2. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  3. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
  4. 2016The Little Engine that Could: Regularization by Denoising (RED)cited by 307
  5. 2022Solving 3D Inverse Problems using Pre-trained 2D Diffusion Modelscited by 132
  6. 2023Delta Denoising Scorecited by 121
  7. 2023Controlling Vision-Language Models for Multi-Task Image Restorationcited by 110
  8. 2023Improving 3D Imaging with Pre-Trained Perpendicular 2D Diffusion Modelscited by 53
  9. 2023Energy-Based Cross Attention for Bayesian Context Update in Text-to-Image Diffusion Modelscited by 30
  10. 2023∞-Diff: Infinite Resolution Diffusion with Subsampled Mollified Statescited by 21
  11. 2023TextIR: A Simple Framework for Text-based Editable Image Restorationcited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 13; 2025: 10; 2026: 8.

32 papers.

Sort:

By area:

  1. 2024A Survey on Diffusion Models for Inverse Problemscited by 180
  2. 2024CFG++: Manifold-constrained Classifier Free Guidance for Diffusion Modelscited by 149
  3. 2025FlowDPS: Flow-Driven Posterior Sampling for Inverse Problemscited by 68
  4. 2024DreamSampler: Unifying Diffusion Sampling and Score Distillation for Image Manipulationcited by 22
  5. 2025SILO: Solving Inverse Problems with Latent Operatorscited by 19
  6. 2024Minority-Focused Text-to-Image Generation via Prompt Optimizationcited by 18
  7. 2025LATINO-PRO: LAtent consisTency INverse sOlver with PRompt Optimizationcited by 16
  8. 2025Diffusion models for inverse problemscited by 16
  9. 2024VISION-XL: High Definition Video Inverse Problem Solver using Latent Image Diffusion Modelscited by 13
  10. 2024Invertible Diffusion Models for Compressed Sensingcited by 12
  11. 2024OmniSSR: Zero-shot Omnidirectional Image Super-Resolution using Stable Diffusion Modelcited by 11
  12. 2024Blind Inversion using Latent Diffusion Priorscited by 10
20 more
  1. 2025Chain-of-Zoom: Extreme Super-Resolution via Scale Autoregression and Preference Alignmentcited by 8
  2. 2025InverseCrafter: Efficient Video ReCapture as a Latent Domain Inverse Problemcited by 6
  3. 2024MRPD: Undersampled MRI reconstruction by prompting a large latent diffusion modelcited by 4
  4. 2025Noise is All You Need: Solving Linear Inverse Problems by Noise Combination Sampling with Diffusion Modelscited by 3
  5. 2023Tell Me What You See: Text-Guided Real-World Image Denoisingcited by 2
  6. 2024Ground-A-Score: Scaling Up the Score Distillation for Multi-Attribute Editingcited by 2
  7. 2024VIPaint: Image Inpainting with Pre-Trained Diffusion Models via Variational Inferencecited by 2
  8. 2026Image Restoration via Diffusion Models with Dynamic Resolutioncited by 2
  9. 2024DifuzCam: Replacing Camera Lens with a Mask and a Diffusion Modelcited by 1
  10. 2025Inference-Time Search Using Side Information for Diffusion-Based Image Reconstructioncited by 1
  11. 2026Measurement-Consistent Langevin Corrector for Stabilizing Latent Diffusion Inverse Problem Solverscited by 1
  12. 2026Language-Free Generative Editing from One Visual Examplecited by 1
  13. 2024ContrastiveCFG: Guiding Diffusion Sampling by Contrasting Positive and Negative Conceptscited by 0
  14. 2025LatentINDIGO: An INN-Guided Latent Diffusion Algorithm for Image Restorationcited by 0
  15. 2025ART-VITON: Measurement-Guided Latent Diffusion for Artifact-Free Virtual Try-Oncited by 0
  16. 2026Active Learning for Conditional Generative Compressed Sensingcited by 0
  17. 2026Triadic Dynamics Aware Diffusion Posterior Sampling for Inverse Problems: Optimizing Guidance and Stochasticity Schedulescited by 0
  18. 2026Zero-Shot Video Restoration and Enhancement with Text-to-Image Latent Diffusion Models and Multi-Modal Referencescited by 0
  19. 2026Generative Priors Conditioned on Natural Language for Bayesian Inversion in PDEscited by 0
  20. 2026BAM! Bayesian Anything Model: a foundation model for generative computational imagingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing and Accelerating Diffusion-Based Inverse Problem Solving through Measurements Optimization
  2. 2023Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion Models
  3. 2026Latent Bias Alignment for High-Fidelity Diffusion Inversion in Real-World Image Reconstruction and Manipulation
  4. 2025Solving Diffusion Inverse Problems with Restart Posterior Sampling
  5. 2025A Gradient Flow Approach to Solving Inverse Problems with Latent Diffusion Models
  6. 2025Unifying and extending Diffusion Models through PDEs for solving Inverse Problems
  7. 2025DAPS++: Rethinking Diffusion Inverse Problems with Decoupled Posterior Annealing
  8. 2025Solving Inverse Problems with FLAIR