# Differentiable Proxy Learning for Adaptive Quantization Control in H.264 Video Coding

arXiv: https://arxiv.org/abs/2607.10478 · HTML: https://mlsanity.com/paper/2607.10478/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 11 Jul 2026.

## Builds on (most cited first, 12 of 12)

- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations](https://mlsanity.com/paper/1704.00648/) (2017)
- [Neural Video Compression with Feature Modulation](https://mlsanity.com/paper/2402.17414/) (2024)
- [Preprocessing Enhanced Image Compression for Machine Vision](https://mlsanity.com/paper/2206.05650/) (2022)
- [Enhanced Standard Compatible Image Compression Framework based on Auxiliary Codec Networks](https://mlsanity.com/paper/2009.14754/) (2020)
- [Sandwiched Compression: Repurposing Standard Codecs with Neural Network Wrappers](https://mlsanity.com/paper/2402.05887/) (2024)
- [How to Exploit the Transferability of Learned Image Compression to Conventional Codecs](https://mlsanity.com/paper/2012.01874/) (2020)
- [Differentiable bit-rate estimation for neural-based video codec enhancement](https://mlsanity.com/paper/2301.09776/) (2023)
- [Deep Video Codec Control for Vision Models](https://mlsanity.com/paper/2308.16215/) (2023)

## Abstract (arXiv)

H.264 has been the most widely used video coding format for the past two decades due to its relative simplicity, efficiency, and wide availability of software and hardware implementations. However, optimizing codec parameters such as the quantization parameter (QP) for specific objectives (e.g., perceptual quality or machine vision tasks) is challenging due to the non-differentiable nature of standard video codecs. While differentiable proxies have recently been used to enable gradient-based optimization around standard codecs, their fidelity to the target codec is rarely explicitly characterized. In this paper, we propose a differentiable proxy learning method for H.264 intra codec to enable adaptive quantization control. Built upon a variable-rate learned compression model, the proposed proxy is made differentiable with respect to codec QP through a soft-indexing mechanism. It is then trained to approximate the rate-distortion behavior of H.264 under two quantization settings: global-QP, which uses one QP per image, and spatial-QP, which assigns QPs at the macroblock level. Using the frozen trained proxy, we develop a proxy-based adaptive quantization (AQ) framework for both perceptual optimization and machine vision tasks. Experimental results demonstrate that the proposed proxies closely approximate the rate-distortion behavior of H.264 intra codec. The resulting proxy-based AQ framework consistently improves rate-task trade-offs over fixed-QP H.264 baselines, achieving BD-rate reduction of up to 17.12% for semantic segmentation and 15.30% for MS-SSIM.