mlsanityData to 14 Oct 2026

VEnhancer: Generative Space-Time Enhancement for Video Generation

arXiv 2407.07667 ↗First posted 10 Jul 2024cs.CV eess.IVcited by 64

On the map: Diffusion models → Video generation

Abstract (arXiv)

We present VEnhancer, a generative space-time enhancement framework that improves the existing text-to-video results by adding more details in spatial domain and synthetic detailed motion in temporal domain. Given a generated low-quality video, our approach can increase its spatial and temporal resolution simultaneously with arbitrary up-sampling space and time scales through a unified video diffusion model. Furthermore, VEnhancer effectively removes generated spatial artifacts and temporal flickering of generated videos. To achieve this, basing on a pretrained video diffusion model, we train a video ControlNet and inject it to the diffusion model as a condition on low frame-rate and low-resolution videos. To effectively train this video ControlNet, we design space-time data augmentation as well as video-aware conditioning. Benefiting from the above designs, VEnhancer yields to be stable during training and shares an elegant end-to-end training manner. Extensive experiments show that VEnhancer surpasses existing state-of-the-art video super-resolution and space-time super-resolution methods in enhancing AI-generated videos. Moreover, with VEnhancer, exisiting open-source state-of-the-art text-to-video method, VideoCrafter-2, reaches the top one in video generation benchmark -- VBench.

Key facts. Cited by 64 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 48 of its 51 references that are in the map. First posted on arXiv 10 Jul 2024.

Builds on

48 of 51 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models17 Jan
  2. 2023Photorealistic Video Generation with Diffusion Models11 Dec
  3. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  4. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models7 Nov
  5. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generation30 Oct
  6. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation27 Sep
  7. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models26 Sep
  8. 2023ModelScope Text-to-Video Technical Report12 Aug
  9. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  10. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  11. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation22 Dec
  12. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct

Other references · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2022Classifier-Free Diffusion Guidancecited by 5,847
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  8. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  9. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  10. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  11. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  12. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
24 more
  1. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
  2. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 950
  3. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  4. 2017Video Enhancement with Task-Oriented Flowcited by 619
  5. 2019EDVR: Video Restoration with Enhanced Deformable Convolutional Networkscited by 494
  6. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  7. 2021BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignmentcited by 342
  8. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 320
  9. 2020BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyondcited by 304
  10. 2023DiffBIR: Towards Blind Image Restoration with Generative Diffusion Priorcited by 257
  11. 2022VRT: A Video Restoration Transformercited by 220
  12. 2022Recurrent Video Restoration Transformer with Guided Deformable Attentioncited by 170
  13. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
  14. 2021Investigating Tradeoffs in Real-World Video Super-Resolutioncited by 134
  15. 2023Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolutioncited by 132
  16. 2021Video Super-Resolution Transformercited by 121
  17. 2020Video Super-Resolution with Recurrent Structure-Detail Networkcited by 116
  18. 2022VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolutioncited by 101
  19. 2020Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolutioncited by 91
  20. 2020Video Super-resolution with Temporal Group Attentioncited by 81
  21. 2020Space-Time-Aware Multi-Resolution Video Enhancementcited by 49
  22. 2020Revisiting Temporal Modeling for Video Super-resolutioncited by 46
  23. 2022Mitigating Artifacts in Real-World Video Super-Resolution Modelscited by 29
  24. 2019FISR: Deep Joint Frame Interpolation and Super-Resolution with a Multi-scale Temporal Losscited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 33; 2026: 24.

64 papers.

Sort:

By area:

  1. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 174
  2. 2024GameGen-X: Interactive Open-world Game Video Generationcited by 128
  3. 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generationcited by 108
  4. 2025Goku: Flow Based Video Generative Foundation Modelscited by 67
  5. 2025PhysGen3D: Crafting a Miniature Interactive World from a Single Imagecited by 66
  6. 2025SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restorationcited by 62
  7. 2025Vchitect-2.0: Parallel Transformer for Scaling Up Video Diffusion Modelscited by 60
  8. 2025STAR: Spatial-Temporal Augmentation with Text-to-Video Models for Real-World Video Super-Resolutioncited by 56
  9. 2024LiFT: Leveraging Human Feedback for Text-to-Video Model Alignmentcited by 53
  10. 2025SeedVR2: One-Step Video Restoration via Diffusion Adversarial Post-Trainingcited by 50
  11. 2025FlashVideo: Flowing Fidelity to Detail for Efficient High-Resolution Video Generationcited by 32
  12. 2025DOVE: Efficient One-Step Diffusion Model for Real-World Video Super-Resolutioncited by 31
38 more
  1. 2024Imagine360: Immersive 360 Video Generation from Perspective Anchorcited by 29
  2. 2025Lux Post Facto: Learning Portrait Performance Relighting with Conditional Video Diffusion and a Hybrid Datasetcited by 28
  3. 2025Beyond the Frame: Generating 360 Panoramic Videos from Perspective Videoscited by 16
  4. 2025PhysCorr: Dual-Reward DPO for Physics-Constrained Text-to-Video Generation with Automated Preference Selectioncited by 15
  5. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  6. 2026LUVE : Latent-Cascaded Ultra-High-Resolution Video Generation with Dual Frequency Expertscited by 15
  7. 2024REDUCIO! Generating 1K Video within 16 Seconds using Extremely Compressed Motion Latentscited by 14
  8. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applicationscited by 14
  9. 2025Turbo2K: Towards Ultra-Efficient and High-Quality 2K Video Synthesiscited by 13
  10. 2025Show and Polish: Reference-Guided Identity Preservation in Face Video Restorationcited by 12
  11. 2025DiffVSR: Revealing an Effective Recipe for Taming Robust Video Super-Resolution Against Complex Degradationscited by 11
  12. 2024Data-Juicer Sandbox: A Feedback-Driven Suite for Multimodal Data-Model Co-developmentcited by 10
  13. 2026CubeComposer: Spatio-Temporal Autoregressive 4K 360deg Video Generation from Perspective Videocited by 10
  14. 2025Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusioncited by 7
  15. 2025LiftVSR: Lifting Image Diffusion to Video Super-Resolution via Hybrid Temporal Modeling with Only 4×RTX 4090scited by 5
  16. 2025SimpleGVR: A Simple Baseline for Latent-Cascaded Video Super-Resolutioncited by 5
  17. 2026Improved Adversarial Diffusion Compression for Real-World Video Super-Resolutioncited by 5
  18. 2025RealisVSR: Detail-enhanced Diffusion for Real-World 4K Video Super-Resolutioncited by 4
  19. 2025HumanSAM: Classifying Human-centric Forgery Videos in Human Spatial, Appearance, and Motion Anomalycited by 4
  20. 2025PatchVSR: Breaking Video Diffusion Resolution Limits with Patch-wise Video Super-Resolutioncited by 4
  21. 2026DUO-VSR: Dual-Stream Distillation for One-Step Video Super-Resolutioncited by 4
  22. 2025OS-DiffVSR: Towards One-step Latent Diffusion Model for High-detailed Real-world Video Super-Resolutioncited by 3
  23. 2025Asymmetric VAE for One-Step Video Super-Resolution Accelerationcited by 3
  24. 2026Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutionscited by 3
  25. 2025TurboVSR: Fantastic Video Upscalers and Where to Find Themcited by 2
  26. 2025QuantVSR: Low-Bit Post-Training Quantization for Real-World Video Super-Resolutioncited by 2
  27. 2025Towards Redundancy Reduction in Diffusion Models for Efficient Video Super-Resolutioncited by 2
  28. 2025Rethinking Diffusion Model-Based Video Super-Resolution: Leveraging Dense Guidance from Aligned Featurescited by 2
  29. 2025CreativeVR: Diffusion-Prior-Guided Approach for Structure and Motion Restoration in Generative and Real Videoscited by 2
  30. 2026SwiftVR: Real-Time One-Step Generative Video Restorationcited by 2
  31. 2026DVG-WM: Disentangled Video Generation Enables Efficient Embodied World Model for Robotic Manipulationcited by 2
  32. 2026Diff-VF: Training-free High-quality Long Video Generation via Diffusion Modelcited by 2
  33. 2025GigaVideo-1: Advancing Video Generation via Automatic Feedback with 4 GPU-Hours Fine-Tuningcited by 1
  34. 2025UniMMVSR: A Unified Multi-Modal Framework for Cascaded Video Super-Resolutioncited by 1
  35. 2026Taming Real-World Space-Time Video Super-Resolution with One-Step Diffusioncited by 1
  36. 2026ChopGrad: Pixel-Wise Losses for Latent Video Diffusion via Truncated Backpropagationcited by 1
  37. 2026PolarVSR: A Unified Framework and Benchmark for Continuous Space-Time Polarization Video Reconstructioncited by 1
  38. 2026TRaM-VSR: Importance-Aware Token Routing and Merging for One-Step Diffusion Video Super-Resolutioncited by 1

50 most cited of 64 All 64 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Lumiere: A Space-Time Diffusion Model for Video Generation
  2. 2022Video Diffusion Models
  3. 2025VGDFR: Diffusion-based Video Generation with Dynamic Latent Frame Rate
  4. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation
  5. 2025SpaceTimePilot: Generative Rendering of Dynamic Scenes Across Space and Time
  6. 2026Image-to-Video Diffusion: From Foundations to Open Frontiers
  7. 2026Adaptive Video Distillation: Mitigating Oversaturation and Temporal Collapse in Few-Step Generation
  8. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generation