# Temporal-Consistent Video Restoration with Pre-trained Diffusion Models

arXiv: https://arxiv.org/abs/2503.14863 · HTML: https://mlsanity.com/paper/2503.14863/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 19 Mar 2025.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 16 of 16)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better](https://mlsanity.com/paper/1908.03826/) (2019)
- [Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation](https://mlsanity.com/paper/2003.13659/) (2020)
- [Generative Diffusion Prior for Unified Image Restoration and Enhancement](https://mlsanity.com/paper/2304.01247/) (2023)
- [ProPainter: Improving Propagation and Transformer for Video Inpainting](https://mlsanity.com/paper/2309.03897/) (2023)
- [Stripformer: Strip Transformer for Fast Image Deblurring](https://mlsanity.com/paper/2204.04627/) (2022)
- [Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problems](https://mlsanity.com/paper/2303.05754/) (2023)
- [Deep Flow-Guided Video Inpainting](https://mlsanity.com/paper/1905.02884/) (2019)
- [DMPlug: A Plug-in Method for Solving Inverse Problems with Diffusion Models](https://mlsanity.com/paper/2405.16749/) (2024)
- [DiffIR2VR-Zero: Zero-Shot Video Restoration with Diffusion-based Image Restoration Models](https://mlsanity.com/paper/2407.01519/) (2024)
- [ID-Blau: Image Deblurring by Implicit Diffusion-based reBLurring AUgmentation](https://mlsanity.com/paper/2312.10998/) (2023)

## Built on by (most cited first, 4 of 4)

- [Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusion](https://mlsanity.com/paper/2512.23709/) (2025)
- [FMPlug: Plug-In Foundation Flow-Matching Priors for Inverse Problems](https://mlsanity.com/paper/2508.00721/) (2025)
- [Saving Foundation Flow-Matching Priors for Inverse Problems](https://mlsanity.com/paper/2511.16520/) (2025)
- [TIGER: Taming Identity, Geometry, and Generative Priors for High-Quality Face Video Restoration](https://mlsanity.com/paper/2606.24336/) (2026)

## Abstract (arXiv)

Video restoration (VR) aims to recover high-quality videos from degraded ones. Although recent zero-shot VR methods using pre-trained diffusion models (DMs) show good promise, they suffer from approximation errors during reverse diffusion and insufficient temporal consistency. Moreover, dealing with 3D video data, VR is inherently computationally intensive. In this paper, we advocate viewing the reverse process in DMs as a function and present a novel Maximum a Posterior (MAP) framework that directly parameterizes video frames in the seed space of DMs, eliminating approximation errors. We also introduce strategies to promote bilevel temporal consistency: semantic consistency by leveraging clustering structures in the seed space, and pixel-level consistency by progressive warping with optical flow refinements. Extensive experiments on multiple virtual reality tasks demonstrate superior visual quality and temporal consistency achieved by our method compared to the state-of-the-art.