# Solving Video Inverse Problems Using Image Diffusion Models

arXiv: https://arxiv.org/abs/2409.02574 · HTML: https://mlsanity.com/paper/2409.02574/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 4 Sep 2024.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 19 of 19)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [Robust Compressed Sensing MRI with Deep Generative Priors](https://mlsanity.com/paper/2108.01368/) (2021)
- [Solving 3D Inverse Problems using Pre-trained 2D Diffusion Models](https://mlsanity.com/paper/2211.10655/) (2022)
- [Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problems](https://mlsanity.com/paper/2303.05754/) (2023)
- [DiffIR2VR-Zero: Zero-Shot Video Restoration with Diffusion-based Image Restoration Models](https://mlsanity.com/paper/2407.01519/) (2024)
- [FMA-Net: Flow-Guided Dynamic Filtering and Iterative Feature Refinement with Multi-Attention for Joint Video Super-Resolution and Deblurring](https://mlsanity.com/paper/2401.03707/) (2024)

## Built on by (most cited first, 14 of 14)

- [Video Depth without Video Models](https://mlsanity.com/paper/2411.19189/) (2024)
- [VISION-XL: High Definition Video Inverse Problem Solver using Latent Image Diffusion Models](https://mlsanity.com/paper/2412.00156/) (2024)
- [InverseCrafter: Efficient Video ReCapture as a Latent Domain Inverse Problem](https://mlsanity.com/paper/2512.05672/) (2025)
- [CCS: Controllable and Constrained Sampling with Diffusion Models via Initial Noise Perturbation](https://mlsanity.com/paper/2502.04670/) (2025)
- [STeP: A Framework for Solving Scientific Video Inverse Problems with Spatiotemporal Diffusion Priors](https://mlsanity.com/paper/2504.07549/) (2025)
- [InstantViR: Real-Time Video Inverse Problem Solver with Distilled Diffusion Prior](https://mlsanity.com/paper/2511.14208/) (2025)
- [Improving 2D Diffusion Models for 3D Medical Imaging with Inter-Slice Consistent Stochasticity](https://mlsanity.com/paper/2602.04162/) (2026)
- [LVTINO: LAtent Video consisTency INverse sOlver for High Definition Video Restoration](https://mlsanity.com/paper/2510.01339/) (2025)
- [MACS: Measurement-Aware Consistency Sampling for Inverse Problems](https://mlsanity.com/paper/2510.02208/) (2025)
- [Improving Temporal Consistency and Fidelity at Inference-time in Perceptual Video Restoration by Zero-shot Image-based Diffusion Models](https://mlsanity.com/paper/2510.25420/) (2025)
- [Zero-Shot Video Restoration and Enhancement with Assistance of Video Diffusion Models](https://mlsanity.com/paper/2601.21922/) (2026)
- [Accelerating Video Inverse Problem Solvers with Autoregressive Diffusion Models](https://mlsanity.com/paper/2605.20624/) (2026)
- [Zero-Shot Video Restoration and Enhancement with Text-to-Image Latent Diffusion Models and Multi-Modal References](https://mlsanity.com/paper/2608.26476/) (2026)
- [LoCoVSR: Local Context Diffusion Posterior Sampling for Video Super-Resolution](https://mlsanity.com/paper/2609.32742/) (2026)

## Abstract (arXiv)

Recently, diffusion model-based inverse problem solvers (DIS) have emerged as state-of-the-art approaches for addressing inverse problems, including image super-resolution, deblurring, inpainting, etc. However, their application to video inverse problems arising from spatio-temporal degradation remains largely unexplored due to the challenges in training video diffusion models. To address this issue, here we introduce an innovative video inverse solver that leverages only image diffusion models. Specifically, by drawing inspiration from the success of the recent decomposed diffusion sampler (DDS), our method treats the time dimension of a video as the batch dimension of image diffusion models and solves spatio-temporal optimization problems within denoised spatio-temporal batches derived from each image diffusion model. Moreover, we introduce a batch-consistent diffusion sampling strategy that encourages consistency across batches by synchronizing the stochastic noise components in image diffusion models. Our approach synergistically combines batch-consistent sampling with simultaneous optimization of denoised spatio-temporal batches at each reverse diffusion step, resulting in a novel and efficient diffusion sampling strategy for video inverse problems. Experimental results demonstrate that our method effectively addresses various spatio-temporal degradations in video inverse problems, achieving state-of-the-art reconstructions. Project page: https://svi-diffusion.github.io/