# VISION-XL: High Definition Video Inverse Problem Solver using Latent Image Diffusion Models

arXiv: https://arxiv.org/abs/2412.00156 · HTML: https://mlsanity.com/paper/2412.00156/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 29 of its 37 references that are in the map.
- First posted on arXiv 29 Nov 2024.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 29 of 29)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models](https://mlsanity.com/paper/2307.00619/) (2023)
- [Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problems](https://mlsanity.com/paper/2303.05754/) (2023)
- [Diffusion Probabilistic Model Made Slim](https://mlsanity.com/paper/2211.17106/) (2022)
- [Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion](https://mlsanity.com/paper/2403.07721/) (2024)
- [Prompt-tuning latent diffusion models for inverse problems](https://mlsanity.com/paper/2310.01110/) (2023)
- [DiffIR2VR-Zero: Zero-Shot Video Restoration with Diffusion-based Image Restoration Models](https://mlsanity.com/paper/2407.01519/) (2024)
- [Regularization by Texts for Latent Diffusion Inverse Solvers](https://mlsanity.com/paper/2311.15658/) (2023)
- [Warped Diffusion: Solving Video Inverse Problems with Image Diffusion Models](https://mlsanity.com/paper/2410.16152/) (2024)
- [Solving Video Inverse Problems Using Image Diffusion Models](https://mlsanity.com/paper/2409.02574/) (2024)
- [Evaluating a Synthetic Image Dataset Generated with Stable Diffusion](https://mlsanity.com/paper/2211.01777/) (2022)

## Built on by (most cited first, 13 of 13)

- [InverseCrafter: Efficient Video ReCapture as a Latent Domain Inverse Problem](https://mlsanity.com/paper/2512.05672/) (2025)
- [STeP: A Framework for Solving Scientific Video Inverse Problems with Spatiotemporal Diffusion Priors](https://mlsanity.com/paper/2504.07549/) (2025)
- [InstantViR: Real-Time Video Inverse Problem Solver with Distilled Diffusion Prior](https://mlsanity.com/paper/2511.14208/) (2025)
- [Improving 2D Diffusion Models for 3D Medical Imaging with Inter-Slice Consistent Stochasticity](https://mlsanity.com/paper/2602.04162/) (2026)
- [Why Do DiT Editors Drift? Plug-and-Play Low Frequency Alignment in VAE Latent Space](https://mlsanity.com/paper/2605.08250/) (2026)
- [LVTINO: LAtent Video consisTency INverse sOlver for High Definition Video Restoration](https://mlsanity.com/paper/2510.01339/) (2025)
- [MACS: Measurement-Aware Consistency Sampling for Inverse Problems](https://mlsanity.com/paper/2510.02208/) (2025)
- [Improving Temporal Consistency and Fidelity at Inference-time in Perceptual Video Restoration by Zero-shot Image-based Diffusion Models](https://mlsanity.com/paper/2510.25420/) (2025)
- [SURE Guided Posterior Sampling: Trajectory Correction for Diffusion-Based Inverse Problems](https://mlsanity.com/paper/2512.23232/) (2025)
- [Zero-Shot Video Restoration and Enhancement with Assistance of Video Diffusion Models](https://mlsanity.com/paper/2601.21922/) (2026)
- [Accelerating Video Inverse Problem Solvers with Autoregressive Diffusion Models](https://mlsanity.com/paper/2605.20624/) (2026)
- [Zero-Shot Video Restoration and Enhancement with Text-to-Image Latent Diffusion Models and Multi-Modal References](https://mlsanity.com/paper/2608.26476/) (2026)
- [LoCoVSR: Local Context Diffusion Posterior Sampling for Video Super-Resolution](https://mlsanity.com/paper/2609.32742/) (2026)

## Abstract (arXiv)

In this paper, we propose a novel framework for solving high-definition video inverse problems using latent image diffusion models. Building on recent advancements in spatio-temporal optimization for video inverse problems using image diffusion models, our approach leverages latent-space diffusion models to achieve enhanced video quality and resolution. To address the high computational demands of processing high-resolution frames, we introduce a pseudo-batch consistent sampling strategy, allowing efficient operation on a single GPU. Additionally, to improve temporal consistency, we present pseudo-batch inversion, an initialization technique that incorporates informative latents from the measurement. By integrating with SDXL, our framework achieves state-of-the-art video reconstruction across a wide range of spatio-temporal inverse problems, including complex combinations of frame averaging and various spatial degradations, such as deblurring, super-resolution, and inpainting. Unlike previous methods, our approach supports multiple aspect ratios (landscape, vertical, and square) and delivers HD-resolution reconstructions (exceeding 1280x720) in under 6 seconds per frame on a single NVIDIA 4090 GPU.