# Multi-rate adaptive transform coding for video compression

arXiv: https://arxiv.org/abs/2210.14308 · HTML: https://mlsanity.com/paper/2210.14308/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Oct 2022.

## Built on by (most cited first, 4 of 4)

- [Computationally-Efficient Neural Image Compression with Shallow Decoders](https://mlsanity.com/paper/2304.06244/) (2023)
- [Adaptive whitening in neural populations with gain-modulating interneurons](https://mlsanity.com/paper/2301.11955/) (2023)
- [Adaptive whitening with fast gain modulation and slow synaptic plasticity](https://mlsanity.com/paper/2308.13633/) (2023)
- [Rate-Adaptive Quantization: A Multi-Rate Codebook Adaptation for Vector Quantization-based Generative Models](https://mlsanity.com/paper/2405.14222/) (2024)

## Abstract (arXiv)

Contemporary lossy image and video coding standards rely on transform coding, the process through which pixels are mapped to an alternative representation to facilitate efficient data compression. Despite impressive performance of end-to-end optimized compression with deep neural networks, the high computational and space demands of these models has prevented them from superseding the relatively simple transform coding found in conventional video codecs. In this study, we propose learned transforms and entropy coding that may either serve as (non)linear drop-in replacements, or enhancements for linear transforms in existing codecs. These transforms can be multi-rate, allowing a single model to operate along the entire rate-distortion curve. To demonstrate the utility of our framework, we augmented the DCT with learned quantization matrices and adaptive entropy coding to compress intra-frame AV1 block prediction residuals. We report substantial BD-rate and perceptual quality improvements over more complex nonlinear transforms at a fraction of the computational cost.