mlsanityData to 14 Oct 2026

Towards Practical Real-Time Neural Video Compression

arXiv 2502.20762 ↗First posted 28 Feb 2025eess.IV cs.CVcited by 80

Not on the map. Why?

Abstract (arXiv)

We introduce a practical real-time neural video codec (NVC) designed to deliver high compression ratio, low latency and broad versatility. In practice, the coding speed of NVCs depends on 1) computational costs, and 2) non-computational operational costs, such as memory I/O and the number of function calls. While most efficient NVCs prioritize reducing computational cost, we identify operational cost as the primary bottleneck to achieving higher coding speed. Leveraging this insight, we introduce a set of efficiency-driven design improvements focused on minimizing operational costs. Specifically, we employ implicit temporal modeling to eliminate complex explicit motion modules, and use single low-resolution latent representations rather than progressive downsampling. These innovations significantly accelerate NVC without sacrificing compression quality. Additionally, we implement model integerization for consistent cross-device coding and a module-bank-based rate control scheme to improve practical adaptability. Experiments show our proposed DCVC-RT achieves an impressive average encoding/decoding speed at 125.2/112.8 fps (frames per second) for 1080p video, while saving an average of 21% in bitrate compared to H.266/VTM. The code is available at https://github.com/microsoft/DCVC.

Key facts. Cited by 80 papers in the map (data through 14 Oct 2026). Builds on 37 of its 54 references that are in the map. First posted on arXiv 28 Feb 2025.

Builds on

37 of 54 references.

References · 37

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 950
  3. 2018Joint Autoregressive and Hierarchical Priors for Learned Image Compressioncited by 664
  4. 2017Video Enhancement with Task-Oriented Flowcited by 619
  5. 2018DVC: An End-to-end Deep Video Compression Frameworkcited by 332
  6. 2017Lossy Image Compression with Compressive Autoencoderscited by 294
  7. 2021Deep Contextual Video Compressioncited by 221
  8. 2023Learned Image Compression with Mixed Transformer-CNN Architecturescited by 211
  9. 2024SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flowcited by 174
  10. 2023Neural Video Compression with Diverse Contextscited by 165
  11. 2022Hybrid Spatial-Temporal Entropy Modelling for Neural Video Compressioncited by 128
  12. 2024Neural Video Compression with Feature Modulationcited by 123
25 more
  1. 2021Temporal Context Mining for Learned Video Compressioncited by 117
  2. 2020M-LVC: Multiple Frames Prediction for Learned Video Compressioncited by 107
  3. 2024GaussianImage: 1000 FPS Image Representation and Compression by 2D Gaussian Splattingcited by 99
  4. 2022VCT: A Video Compression Transformercited by 74
  5. 2021ELF-VC: Efficient Learned Flexible-Rate Video Codingcited by 69
  6. 2020Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Modelcited by 67
  7. 2022Coarse-to-fine Deep Video Coding with Hyperprior-guided Mode Predictioncited by 55
  8. 2023C3: High-performance and low-complexity neural compression from a single image or videocited by 55
  9. 2023EVC: Towards Real-Time Neural Image Compression with Mask Decaycited by 49
  10. 2022CANF-VC: Conditional Augmented Normalizing Flows for Video Compressioncited by 46
  11. 2020Conditional Entropy Coding for Efficient Video Compressioncited by 42
  12. 2021Overfitting for Fun and Profit: Instance-Adaptive Data Compressioncited by 38
  13. 2022AlphaVC: High-Performance and Efficient Learned Video Compressioncited by 28
  14. 2021Conditional Coding for Flexible Learned Video Compressioncited by 26
  15. 2023Computationally-Efficient Neural Image Compression with Shallow Decoderscited by 23
  16. 2022MobileCodec: Neural Inter-frame Video Compression on Mobile Devicescited by 19
  17. 2024PNVC: Towards Practical INR-based Video Compressioncited by 19
  18. 2022Post-Training Quantization for Cross-Platform Learned Image Compressioncited by 14
  19. 2023Hierarchical B-frame Video Coding Using Two-Layer CANF without Motion Codingcited by 14
  20. 2023MobileNVC: Real-time 1080p Neural Video Compression on a Mobile Devicecited by 13
  21. 2023MMVC: Learned Multi-Mode Video Compression with Block-based Prediction Mode Selection and Density-Adaptive Entropy Codingcited by 9
  22. 2023Towards Real-Time Neural Video Codec for Cross-Platform Application Using Calibration Informationcited by 9
  23. 2023Deep Hierarchical Video Compressioncited by 9
  24. 2022Boosting neural video codecs by exploiting hierarchical redundancycited by 5
  25. 2024High-Efficiency Neural Video Compression via Hierarchical Predictive Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 21; 2026: 59.

80 papers.

Sort:

By area:

  1. 2025Learned Image Compression with Hierarchical Progressive Context Modelingcited by 21
  2. 2025Single-step Diffusion-based Video Coding with Semantic-Temporal Guidancecited by 13
  3. 2025Generative Latent Video Compressioncited by 11
  4. 2026Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Modelcited by 10
  5. 2025Ultra-Low Bitrate Perceptual Image Compression with Shallow Encodercited by 7
  6. 2026CoD-Lite: Real-Time Diffusion-Based Generative Image Compressioncited by 7
  7. 2026Ultra-Fast Neural Video Compressioncited by 6
  8. 2025Conditional Video Generation for High-Efficiency Video Compressioncited by 5
  9. 2025Neural B-frame Video Compression with Bi-directional Reference Harmonizationcited by 5
  10. 2026DiT-IC: Aligned Diffusion Transformer for Efficient Image Compressioncited by 5
  11. 2026GVCC: Zero-Shot Video Compression via Codebook-Driven Stochastic Rectified Flowcited by 5
  12. 2025Real-Time Neural Video Compression with Unified Intra and Inter Codingcited by 4
38 more
  1. 2025Low-Bitrate Video Compression through Semantic-Conditioned Diffusioncited by 4
  2. 2026Generative Video Compression with One-Dimensional Latent Representationcited by 4
  3. 2025Revisiting MLLM Token Technology through the Lens of Classical Visual Codingcited by 3
  4. 2025When MLLMs Meet Compression Distortion: A Coding Paradigm Tailored to MLLMscited by 3
  5. 2026DiffVC-RT: Towards Practical Real-Time Diffusion-based Perceptual Neural Video Compressioncited by 3
  6. 2026ProGIC: Progressive and Lightweight Generative Image Compression with Residual Vector Quantizationcited by 3
  7. 2026Enhancing Neural Video Compression of Static Scenes with Positive-Incentive Noisecited by 3
  8. 2026What Matters in Practical Learned Image Compressioncited by 3
  9. 2026ZeroGVC: Zero-Shot Generative Video Compression with Autoregressive Diffusion Priorscited by 3
  10. 2026Enhanced Neural Video Representation Compression Across Extreme Complexity and Quality Scalescited by 3
  11. 2026ECoNGS: Efficient Compressive Neural Gaussian Splats for Volume Visualizationcited by 3
  12. 2025Neural Stereo Video Compression with Hybrid Disparity Compensationcited by 2
  13. 2025SIEDD: Shared-Implicit Encoder with Discrete Decoderscited by 2
  14. 2025Scaling Learned Image Compression Models up to 1 Billioncited by 2
  15. 2026Compression Tells Intelligence: Visual Coding, Visual Token Technology, and the Unificationcited by 2
  16. 2026Leveraging Second-Order Curvature for Efficient Learned Image Compression: Theory and Empirical Evidencecited by 2
  17. 2026TaCo: A Benchmark for Lossless and Lossy Codecs of Heterogeneous Tactile Datacited by 2
  18. 2026Compression as Adaptation: Implicit Visual Representation with Diffusion Foundation Modelscited by 2
  19. 2025Accelerating Learned Image Compression Through Modeling Neural Training Dynamicscited by 1
  20. 2025Neural Weight Compression for Language Modelscited by 1
  21. 2025Neural Compression of 360-Degree Equirectangular Videos using Quality Parameter Adaptationcited by 1
  22. 2025NeRV360: Neural Representation for 360-Degree Videos with a Viewport Decodercited by 1
  23. 2026TeCoNeRV: Leveraging Temporal Coherence for Compressible Neural Representations for Videoscited by 1
  24. 2026Uni-LVC: A Unified Method for Intra- and Inter-Mode Learned Video Compressioncited by 1
  25. 2026Towards Practical Lossless Neural Compression for LiDAR Point Cloudscited by 1
  26. 2026Neural Video Compression with Domain Transfercited by 1
  27. 2026How Accurate are Video Quality Models for Diffusion-Based Video Super-Resolution?cited by 1
  28. 2026Learned Image Compression for Vision-Language-Action Modelscited by 1
  29. 2026MLVC: Multi-platform Learned Video Codec for Real-World Deploymentcited by 1
  30. 2026Generative Video Compression with Adaptive Score Distillationcited by 1
  31. 2026GVC-RT: Towards Real-Time Generative Video Compression at Ultra-Low Bitratescited by 1
  32. 2025PerCoV2: Ultra-Low Bit-Rate Perceptual Image Compression via Query-Based 1D Multimodal Image Tokenscited by 0
  33. 2025Bidirectional Feature-aligned Motion Transformation for Efficient Dynamic Point Cloud Compressioncited by 0
  34. 2025Task-Oriented Communications for 3D Scene Representation: Balancing Timeliness and Fidelitycited by 0
  35. 2025Neural B-Frame Coding: Tackling Domain Shift Issues with Lightweight Online Motion Resolution Adaptationcited by 0
  36. 2026SCENE: Semantic-aware Codec Enhancement with Neural Embeddingscited by 0
  37. 2026A Causal Diffusion Model for Video Reconstruction from Ultra-Low-Bitrate Representationscited by 0
  38. 2026LG-HCC: Local Geometry-Aware Hierarchical Context Compression for 3D Gaussian Splattingcited by 0

50 most cited of 80 All 80 (JSON)