# Generalized Octave Convolutions for Learned Multi-Frequency Image Compression

arXiv: https://arxiv.org/abs/2002.10032 · HTML: https://mlsanity.com/paper/2002.10032/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Feb 2020.

## Built on by (most cited first, 6 of 6)

- [End-to-end Neural Video Coding Using a Compound Spatiotemporal Representation](https://mlsanity.com/paper/2108.04103/) (2021)
- [Toward Real-World Light Field Super-Resolution](https://mlsanity.com/paper/2305.18994/) (2023)
- [Recognition-Aware Learned Image Compression](https://mlsanity.com/paper/2202.00198/) (2022)
- [End-to-End Learned Block-Based Image Compression with Block-Level Masked Convolutions and Asymptotic Closed Loop Training](https://mlsanity.com/paper/2203.11686/) (2022)
- [End-to-End Optimized Image Compression with the Frequency-Oriented Transform](https://mlsanity.com/paper/2401.08194/) (2024)
- [A Modular System for Enhanced Robustness of Multimedia Understanding Networks via Deep Parametric Estimation](https://mlsanity.com/paper/2402.18402/) (2024)

## Abstract (arXiv)

Learned image compression has recently shown the potential to outperform the standard codecs. State-of-the-art rate-distortion (R-D) performance has been achieved by context-adaptive entropy coding approaches in which hyperprior and autoregressive models are jointly utilized to effectively capture the spatial dependencies in the latent representations. However, the latents are feature maps of the same spatial resolution in previous works, which contain some redundancies that affect the R-D performance. In this paper, we propose the first learned multi-frequency image compression and entropy coding approach that is based on the recently developed octave convolutions to factorize the latents into high and low frequency (resolution) components, where the low frequency is represented by a lower resolution. Therefore, its spatial redundancy is reduced, which improves the R-D performance. Novel generalized octave convolution and octave transposed-convolution architectures with internal activation layers are also proposed to preserve more spatial structure of the information. Experimental results show that the proposed scheme not only outperforms all existing learned methods as well as standard codecs such as the next-generation video coding standard VVC (4:2:0) on the Kodak dataset in both PSNR and MS-SSIM. We also show that the proposed generalized octave convolution can improve the performance of other auto-encoder-based computer vision tasks such as semantic segmentation and image denoising.