mlsanityData to 14 Oct 2026

Deep Video Codec Control for Vision Models

arXiv 2308.16215 ↗First posted 30 Aug 2023eess.IV cs.CV cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Standardized lossy video coding is at the core of almost all real-world video processing pipelines. Rate control is used to enable standard codecs to adapt to different network bandwidth conditions or storage constraints. However, standard video codecs (e.g., H.264) and their rate control modules aim to minimize video distortion w.r.t. human quality assessment. We demonstrate empirically that standard-coded videos vastly deteriorate the performance of deep vision models. To overcome the deterioration of vision performance, this paper presents the first end-to-end learnable deep video codec control that considers both bandwidth constraints and downstream deep vision performance, while adhering to existing standardization. We demonstrate that our approach better preserves downstream deep vision performance than traditional standard video coding.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 37 of its 77 references that are in the map. First posted on arXiv 30 Aug 2023.

Builds on

37 of 77 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  7. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  8. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  9. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  10. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  11. 2020Knowledge Distillation: A Surveycited by 1,603
  12. 2018Group Normalizationcited by 1,524
25 more
  1. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  2. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 637
  3. 2015Empirical Evaluation of Rectified Activations in Convolutional Networkcited by 519
  4. 2018Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transformcited by 371
  5. 2019Kornia: an Open Source Differentiable Computer Vision Library for PyTorchcited by 232
  6. 2020Focal Frequency Loss for Image Reconstruction and Synthesiscited by 206
  7. 2017Modulating early visual processing by languagecited by 167
  8. 2021Self-supervised Augmentation Consistency for Adapting Semantic Segmentationcited by 149
  9. 2015Delving Deeper into Convolutional Networks for Learning Video Representationscited by 148
  10. 2017Backpropagation through the Void: Optimizing control variates for black-box gradient estimationcited by 106
  11. 2020Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analyticscited by 80
  12. 2021PyTorchVideo: A Deep Learning Library for Video Understandingcited by 32
  13. 2022MuZero with Self-competition for Rate Control in VP9 Video Compressioncited by 28
  14. 2020The Rate-Distortion-Accuracy Tradeoff: JPEG Case Studycited by 22
  15. 2021Self-Conditioned Probabilistic Learning of Video Rescalingcited by 19
  16. 2021Recent Standard Development Activities on Video Coding for Machinescited by 14
  17. 2023DistractFlow: Improving Optical Flow Estimation via Realistic Distractions and Pseudo-Labelingcited by 14
  18. 2021A Survey on Perceptually Optimized Video Codingcited by 10
  19. 2020An Emerging Coding Paradigm VCM: A Scalable Coding Approach Beyond Feature and Signalcited by 8
  20. 2023Edge-Based Video Analytics: A Surveycited by 8
  21. 2023Sandwiched Video Compression: Efficiently Extending the Reach of Standard Codecs with Neural Wrapperscited by 6
  22. 2020How to Exploit the Transferability of Learned Image Compression to Conventional Codecscited by 5
  23. 2017Learning a Virtual Codec Based on Deep Convolutional Neural Network to Compress Imagecited by 4
  24. 2022Performance Evaluation of Action Recognition Models on Low Quality Videoscited by 3
  25. 2022Task Oriented Video Coding: A Surveycited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 1.

2 papers.

  1. 2024Transformer-Aided Semantic Communicationscited by 3
  2. 2026Differentiable Proxy Learning for Adaptive Quantization Control in H.264 Video Codingcited by 0