mlsanityData to 7 Oct 2026

Zero-Shot Video Restoration and Enhancement with Text-to-Image Latent Diffusion Models and Multi-Modal References

arXiv 2608.26476 ↗First posted 27 Aug 2026cs.CVcited by 0

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

Zero-shot image restoration methods with text-to-image latent diffusion models have achieved great success in universal image restoration tasks without training. However, applying them to video restoration will result in severe temporal flickering. In this paper, we propose a novel framework for zero-shot video restoration and enhancement which uses a text-to-image latent diffusion model and multi-modal references. Through the proposed dual prompt tuning inversion and sampling, the inference time can be reduced to nearly 1/3 of the original. The performance and temporal consistency can be also significantly stregthened. By using the proposed texture-aware video token merging, the temporal correlation between frames can be further utilized to improve the temporal consistency. We futher propose the referenced self-attention and referenced token merging to support image reference. Experimental results demonstrate the superiority of the proposed method in restoring and enhancing temporally consistent videos.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 36 of its 40 references that are in the map. First posted on arXiv 27 Aug 2026.

Builds on

36 of 40 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024VISION-XL: High Definition Video Inverse Problem Solver using Latent Image Diffusion Models29 Nov
  2. 2024Solving Video Inverse Problems Using Image Diffusion Models4 Sep
  3. 2024Zero-Shot Video Restoration and Enhancement Using Pre-Trained Image Diffusion Model2 Jul
  4. 2024DiffIR2VR-Zero: Zero-Shot Video Restoration with Diffusion-based Image Restoration Models1 Jul
  5. 2023Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusion1 Dec
  6. 2023Regularization by Texts for Latent Diffusion Inverse Solvers27 Nov
  7. 2023Prompt-tuning latent diffusion models for inverse problems2 Oct
  8. 2023Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models2 Jul
  9. 2023Accelerating Diffusion Models for Inverse Problems through Shortcut Sampling26 May
  10. 2023Generative Diffusion Prior for Unified Image Restoration and Enhancement3 Apr
  11. 2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model1 Dec
  12. 2022Diffusion Posterior Sampling for General Noisy Inverse Problems29 Sep
2 more
  1. 2022Denoising Diffusion Restoration Models27 Jan
  2. 2021Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction9 Dec

Other references · 22

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2022Classifier-Free Diffusion Guidancecited by 5,717
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  7. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  8. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  9. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  10. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  11. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  12. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
10 more
  1. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  2. 2018Learning Blind Video Temporal Consistencycited by 247
  3. 2023Token Merging for Fast Stable Diffusioncited by 220
  4. 2023VidToMe: Video Token Merging for Zero-Shot Video Editingcited by 111
  5. 2021ClassSR: A General Framework to Accelerate Super-Resolution Networks by Data Characteristiccited by 81
  6. 2019Deep Exemplar-based Video Colorizationcited by 75
  7. 2022Reference-based Video Super-Resolution Using Multi-Camera Video Tripletscited by 19
  8. 2024ReFIR: Grounding Large Restoration Models with Retrieval Augmentationcited by 15
  9. 2022Reference-based Image and Video Super-Resolution via C2-Matchingcited by 8
  10. 2024Accelerating Image Super-Resolution Networks with Pixel-Level Classificationcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Zero-Shot Video Restoration and Enhancement with Assistance of Video Diffusion Models
  2. 2025DiTVR: Zero-Shot Diffusion Transformer for Video Restoration
  3. 2025TDM: Temporally-Consistent Diffusion Model for All-in-One Real-World Video Restoration
  4. 2023Zero-Shot Video Editing Using Off-The-Shelf Image Diffusion Models
  5. 2025Improving Temporal Consistency and Fidelity at Inference-time in Perceptual Video Restoration by Zero-shot Image-based Diffusion Models
  6. 2025LVTINO: LAtent Video consisTency INverse sOlver for High Definition Video Restoration
  7. 2024TI2V-Zero: Zero-Shot Image Conditioning for Text-to-Video Diffusion Models
  8. 2023Unlimited-Size Diffusion Restoration