mlsanityData to 14 Oct 2026

Neural Video Compression using GANs for Detail Synthesis and Propagation

arXiv 2107.12038 ↗First posted 26 Jul 2021eess.IV cs.CVcited by 36

Not on the map. Why?

Abstract (arXiv)

We present the first neural video compression method based on generative adversarial networks (GANs). Our approach significantly outperforms previous neural and non-neural video compression methods in a user study, setting a new state-of-the-art in visual quality for neural methods. We show that the GAN loss is crucial to obtain this high visual quality. Two components make the GAN loss effective: we i) synthesize detail by conditioning the generator on a latent extracted from the warped previous reconstruction to then ii) propagate this detail with high-quality flow. We find that user studies are required to compare methods, i.e., none of our quantitative metrics were able to predict all studies. We present the network design choices in detail, and ablate them with user studies.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Builds on 27 of its 52 references that are in the map. First posted on arXiv 26 Jul 2021.

Builds on

27 of 52 references.

References · 27

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2014Conditional Generative Adversarial Netscited by 2,162
  3. 2018Variational image compression with a scale hyperpriorcited by 935
  4. 2018Joint Autoregressive and Hierarchical Priors for Learned Image Compressioncited by 664
  5. 2017The Perception-Distortion Tradeoffcited by 587
  6. 2018Video-to-Video Synthesiscited by 390
  7. 2020High-Fidelity Generative Image Compressioncited by 338
  8. 2018DVC: An End-to-end Deep Video Compression Frameworkcited by 332
  9. 2020Channel-wise Autoregressive Entropy Models for Learned Image Compressioncited by 264
  10. 2018Generative Adversarial Networks for Extreme Learned Image Compressioncited by 219
  11. 2019Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoffcited by 211
  12. 2018Video Compression through Image Interpolationcited by 143
15 more
  1. 2017Real-Time Adaptive Image Compressioncited by 130
  2. 2020What Matters in Unsupervised Optical Flowcited by 124
  3. 2019Video Compression With Rate-Distortion Autoencoderscited by 97
  4. 2018Learned Video Compressioncited by 89
  5. 2021ELF-VC: Efficient Learned Flexible-Rate Video Codingcited by 69
  6. 2020Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Modelcited by 67
  7. 2018Deep Generative Models for Distribution-Preserving Lossy Compressioncited by 59
  8. 2020Conditional Entropy Coding for Efficient Video Compressioncited by 42
  9. 2017Generative Compressioncited by 40
  10. 2020Feedback Recurrent Autoencoder for Video Compressioncited by 30
  11. 2021A coding theorem for the rate-distortion-perception functioncited by 30
  12. 2020Hierarchical Autoregressive Modeling for Neural Video Compressioncited by 28
  13. 2021On the advantages of stochastic encoderscited by 28
  14. 2019Neural Video Compression using Spatio-Temporal Priorscited by 7
  15. 2020Lossy Compression with Distortion Constrained Optimizationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 2; 2023: 8; 2024: 7; 2025: 4; 2026: 13.

36 papers.

Sort:

By area:

  1. 2021Temporal Context Mining for Learned Video Compressioncited by 117
  2. 2023Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Modelscited by 94
  3. 2022VCT: A Video Compression Transformercited by 74
  4. 2023PADL: Language-Directed Physics-Based Character Controlcited by 48
  5. 2023Interactive Face Video Coding: A Generative Compression Frameworkcited by 17
  6. 2021Learning Cross-Scale Weighted Prediction for Efficient Neural Video Compressioncited by 13
  7. 2024Generative Visual Compression: A Reviewcited by 13
  8. 2025Single-step Diffusion-based Video Coding with Semantic-Temporal Guidancecited by 13
  9. 2025Generative Latent Video Compressioncited by 11
  10. 2023On the Choice of Perception Loss Function for Learned Video Compressioncited by 10
  11. 2024M3-CVC: Controllable Video Compression with Multimodal Generative Modelscited by 10
  12. 2023High Visual-Fidelity Learned Video Compressioncited by 8
24 more
  1. 2024Rate-Distortion-Perception Tradeoff Based on the Conditional-Distribution Perception Measurecited by 8
  2. 2024Extreme Video Compression with Pre-trained Diffusion Modelscited by 6
  3. 2024Task-Aware Encoder Control for Deep Video Compressioncited by 6
  4. 2022Boosting neural video codecs by exploiting hierarchical redundancycited by 5
  5. 2026GVCC: Zero-Shot Video Compression via Codebook-Driven Stochastic Rectified Flowcited by 5
  6. 2026ProGVC: Progressive-based Generative Video Compression via Auto-Regressive Context Modelingcited by 3
  7. 2026ZeroGVC: Zero-Shot Generative Video Compression with Autoregressive Diffusion Priorscited by 3
  8. 2023Learnt Deep Hyperparameter selection in Adversarial Training for compressed video enhancement with perceptual criticcited by 2
  9. 2024Survey on Visual Signal Coding and Processing with Generative Models: Technologies, Standards and Optimizationcited by 2
  10. 2025Higher fidelity perceptual image and video compression with a latent conditioned residual denoising diffusion modelcited by 1
  11. 2026Efficient Video Diffusion with Sparse Information Transmission for Video Compressioncited by 1
  12. 2026Dense-Prior-Guided Generative Video Compressioncited by 1
  13. 2026MLVC: Multi-platform Learned Video Codec for Real-World Deploymentcited by 1
  14. 2026Generative Video Compression with Adaptive Score Distillationcited by 1
  15. 2023Context-Aware Neural Video Compression on Solar Dynamics Observatorycited by 0
  16. 2023Blurry Video Compression: A Trade-off between Visual Enhancement and Data Compressioncited by 0
  17. 2024Neural-based Video Compression on Solar Dynamics Observatory Imagescited by 0
  18. 2025On Self-Adaptive Perception Loss Function for Sequential Lossy Compressioncited by 0
  19. 2026Deep Joint Source-Channel Coding for Wireless Video Transmission with Asymmetric Contextcited by 0
  20. 2026Geometric Transformation-Embedded Mamba for Learned Video Compressioncited by 0
  21. 2026ENCORE: Event-Assisted Complementary Motion Refinement for Learned Video Compressioncited by 0
  22. 2026CodecArena: Codec Quality Assessment via Visual Reinforcement Learningcited by 0
  23. 2026Generative Video Compression Based on Hierarchical Referencingcited by 0
  24. 2026A Spatiotemporal Semantic Importance-Guided Unified Compression and Editing Framework for AI-Generated Videoscited by 0