mlsanityData to 7 Oct 2026

Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution

arXiv 2312.06640 ↗First posted 11 Dec 2023cs.CVcited by 131

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

Text-based diffusion models have exhibited remarkable success in generation and editing, showing great promise for enhancing visual content with their generative prior. However, applying these models to video super-resolution remains challenging due to the high demands for output fidelity and temporal consistency, which is complicated by the inherent randomness in diffusion models. Our study introduces Upscale-A-Video, a text-guided latent diffusion framework for video upscaling. This framework ensures temporal coherence through two key mechanisms: locally, it integrates temporal layers into U-Net and VAE-Decoder, maintaining consistency within short sequences; globally, without training, a flow-guided recurrent latent propagation module is introduced to enhance overall video stability by propagating and fusing latent across the entire sequences. Thanks to the diffusion paradigm, our model also offers greater flexibility by allowing text prompts to guide texture creation and adjustable noise levels to balance restoration and generation, enabling a trade-off between fidelity and quality. Extensive experiments show that Upscale-A-Video surpasses existing methods in both synthetic and real-world benchmarks, as well as in AI-generated videos, showcasing impressive visual realism and temporal consistency.

Key facts. Cited by 131 papers in the map (data through 7 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 75 of its 84 references that are in the map. First posted on arXiv 11 Dec 2023.

Builds on

75 of 84 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance19 Sep
  2. 2023DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior29 Aug
  3. 2023Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization28 Aug
  4. 2023ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting23 Jul
  5. 2023Exploiting Diffusion Prior for Real-World Image Super-Resolution11 May
  6. 2023DiffIR: Efficient Diffusion Model for Image Restoration16 Mar
  7. 2023Denoising Diffusion Probabilistic Models for Robust Image Super-Resolution in the Wild15 Feb
  8. 2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model1 Dec
  9. 2022Improving Diffusion Models for Inverse Problems using Manifold Constraints2 Jun
  10. 2022Denoising Diffusion Restoration Models27 Jan

Other references · 65

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 2022Classifier-Free Diffusion Guidancecited by 5,717
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
38 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  2. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  3. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  4. 2022Video Diffusion Modelscited by 2,280
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  6. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  7. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  8. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  9. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  10. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  11. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  12. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 974
  13. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 961
  14. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  15. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  16. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  17. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  18. 2017Video Enhancement with Task-Oriented Flowcited by 612
  19. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  20. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  21. 2019EDVR: Video Restoration with Enhanced Deformable Convolutional Networkscited by 492
  22. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  23. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
  24. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  25. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  26. 2018Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transformcited by 369
  27. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  28. 2021BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignmentcited by 340
  29. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  30. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 314
  31. 2020BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyondcited by 303
  32. 2022Perception Prioritized Training of Diffusion Modelscited by 301
  33. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 300
  34. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 284
  35. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  36. 2018Learning Blind Video Temporal Consistencycited by 247
  37. 2022VRT: A Video Restoration Transformercited by 219
  38. 2017UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Losscited by 202

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 15; 2025: 67; 2026: 48.

131 papers.

Sort:

By area:

  1. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
  2. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
  3. 20243DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priorscited by 106
  4. 2024VEnhancer: Generative Space-Time Enhancement for Video Generationcited by 64
  5. 2025SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restorationcited by 62
  6. 2025STAR: Spatial-Temporal Augmentation with Text-to-Video Models for Real-World Video Super-Resolutioncited by 56
  7. 2024Improved Video VAE for Latent Video Diffusion Modelcited by 46
  8. 2025SeedVR2: One-Step Video Restoration via Diffusion Adversarial Post-Trainingcited by 45
  9. 2025FlashVSR: Towards Real-Time Diffusion-Based Streaming Video Super-Resolutioncited by 40
  10. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  11. 2024DiffIR2VR-Zero: Zero-Shot Video Restoration with Diffusion-based Image Restoration Modelscited by 33
  12. 2025FlashVideo: Flowing Fidelity to Detail for Efficient High-Resolution Video Generationcited by 32
38 more
  1. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  2. 2024VideoGigaGAN: Towards Detail-rich Video Super-Resolutioncited by 30
  3. 2025DOVE: Efficient One-Step Diffusion Model for Real-World Video Super-Resolutioncited by 30
  4. 2025Lux Post Facto: Learning Portrait Performance Relighting with Conditional Video Diffusion and a Hybrid Datasetcited by 28
  5. 2025MatAnyone: Stable Video Matting with Consistent Memory Propagationcited by 24
  6. 2024SuperGaussian: Repurposing Video Models for 3D Super Resolutioncited by 22
  7. 2025UltraGen: High-Resolution Video Generation with Hierarchical Attentioncited by 17
  8. 2025DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencodercited by 15
  9. 2025Alias-Free Latent Diffusion Models: Improving Fractional Shift Equivariance of Diffusion Latent Spacecited by 14
  10. 2025One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolutioncited by 14
  11. 2025Turbo2K: Towards Ultra-Efficient and High-Quality 2K Video Synthesiscited by 13
  12. 20243DEnhancer: Consistent Multi-View Diffusion for 3D Enhancementcited by 12
  13. 2025MoA-VR: A Mixture-of-Agents System Towards All-in-One Video Restorationcited by 12
  14. 2025DiffVSR: Revealing an Effective Recipe for Taming Robust Video Super-Resolution Against Complex Degradationscited by 11
  15. 2025Enhance-A-Video: Better Generated Video for Freecited by 11
  16. 2025Vivid-VR: Distilling Concepts from Text-to-Video Diffusion Transformer for Photorealistic Video Restorationcited by 10
  17. 2025Training-free Mixed-Resolution Latent Upsampling for Spatially Accelerated Diffusion Transformerscited by 9
  18. 2023TMP: Temporal Motion Propagation for Online Video Super-Resolutioncited by 8
  19. 2025InfVSR: Toward Consistency-Driven Streaming Generative Video Super-Resolutioncited by 8
  20. 2025HiStream: Efficient High-Resolution Video Generation via Redundancy-Eliminated Streamingcited by 8
  21. 2026AdaGaR: Adaptive Gabor Representation for Dynamic Scene Reconstructioncited by 8
  22. 2024NaRCan: Natural Refined Canonical Image with Integration of Diffusion Prior for Video Editingcited by 7
  23. 2025VideoFusion: A Spatio-Temporal Collaborative Network for Multi-modal Video Fusioncited by 7
  24. 2025DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Spacecited by 7
  25. 2025Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusioncited by 7
  26. 2025FreeSwim: Revisiting Sliding-Window Attention Mechanisms for Training-Free Ultra-High-Resolution Video Generationcited by 6
  27. 2026RoboVIP: Multi-View Video Generation with Visual Identity Prompting Augments Robot Manipulationcited by 6
  28. 2026S2DiT: Sandwich Diffusion Transformer for Mobile Streaming Video Generationcited by 6
  29. 2026VAnim: Rendering-Aware Sparse State Modeling for Structure-Preserving Vector Animationcited by 6
  30. 2025SIR-DIFF: Sparse Image Sets Restoration with Multi-View Diffusion Modelcited by 5
  31. 2025Self-supervised ControlNet with Spatio-Temporal Mamba for Real-world Video Super-resolutioncited by 5
  32. 2025Restereo: Diffusion stereo video generation and restorationcited by 5
  33. 2025SimpleGVR: A Simple Baseline for Latent-Cascaded Video Super-Resolutioncited by 5
  34. 2025DAM-VSR: Disentanglement of Appearance and Motion for Video Super-Resolutioncited by 5
  35. 2025Masked Temporal Interpolation Diffusion for Procedure Planning in Instructional Videoscited by 5
  36. 2025Bridging Diffusion Models and 3D Representations: A 3D Consistent Super-Resolution Frameworkcited by 5
  37. 2025Point Prompting: Counterfactual Tracking with Video Diffusion Modelscited by 5
  38. 2025IC-World: In-Context Generation for Shared World Modelingcited by 5

50 most cited of 131 All 131 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Motion-Guided Latent Diffusion for Temporally Consistent Real-world Video Super-resolution
  2. 2024Inflation with Diffusion: Efficient Temporal Adaptation for Text-to-Video Super-Resolution
  3. 2023VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation
  4. 2023Enhancing Perceptual Quality in Video Super-Resolution through Temporally-Consistent Detail Synthesis using Diffusion Models
  5. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
  6. 2023Photorealistic Video Generation with Diffusion Models
  7. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation
  8. 2026Efficient Video Diffusion Models: Advancements and Challenges