# End-to-End Optimized Image Compression with the Frequency-Oriented Transform

arXiv: https://arxiv.org/abs/2401.08194 · HTML: https://mlsanity.com/paper/2401.08194/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 16 Jan 2024.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [CompressAI: a PyTorch library and evaluation platform for end-to-end compression research](https://mlsanity.com/paper/2011.03029/) (2020)
- [Variable Rate Image Compression with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06085/) (2015)
- [Context-adaptive Entropy Model for End-to-end Optimized Image Compression](https://mlsanity.com/paper/1809.10452/) (2018)
- [Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution](https://mlsanity.com/paper/1904.05049/) (2019)
- [Real-Time Adaptive Image Compression](https://mlsanity.com/paper/1705.05823/) (2017)
- [Variable Rate Deep Image Compression With a Conditional Autoencoder](https://mlsanity.com/paper/1909.04802/) (2019)
- [Non-local Attention Optimized Deep Image Compression](https://mlsanity.com/paper/1904.09757/) (2019)
- [The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies](https://mlsanity.com/paper/2007.15911/) (2020)
- [Learning End-to-End Lossy Image Compression: A Benchmark](https://mlsanity.com/paper/2002.03711/) (2020)
- [Learning Omni-frequency Region-adaptive Representations for Real Image Super-Resolution](https://mlsanity.com/paper/2012.06131/) (2020)
- [Multi-Scale Single Image Dehazing Using Laplacian and Gaussian Pyramids](https://mlsanity.com/paper/2111.05700/) (2021)
- [Neural Multi-scale Image Compression](https://mlsanity.com/paper/1805.06386/) (2018)
- [Layered Image Compression using Scalable Auto-encoder](https://mlsanity.com/paper/1904.00553/) (2019)
- [Generalized Octave Convolutions for Learned Multi-Frequency Image Compression](https://mlsanity.com/paper/2002.10032/) (2020)

## Built on by (most cited first, 2 of 2)

- [Approximately Invertible Neural Network for Learned Image Compression](https://mlsanity.com/paper/2408.17073/) (2024)
- [Rethinking Learned Image Compression: Context is All You Need](https://mlsanity.com/paper/2407.11590/) (2024)

## Abstract (arXiv)

Image compression constitutes a significant challenge amidst the era of information explosion. Recent studies employing deep learning methods have demonstrated the superior performance of learning-based image compression methods over traditional codecs. However, an inherent challenge associated with these methods lies in their lack of interpretability. Following an analysis of the varying degrees of compression degradation across different frequency bands, we propose the end-to-end optimized image compression model facilitated by the frequency-oriented transform. The proposed end-to-end image compression model consists of four components: spatial sampling, frequency-oriented transform, entropy estimation, and frequency-aware fusion. The frequency-oriented transform separates the original image signal into distinct frequency bands, aligning with the human-interpretable concept. Leveraging the non-overlapping hypothesis, the model enables scalable coding through the selective transmission of arbitrary frequency components. Extensive experiments are conducted to demonstrate that our model outperforms all traditional codecs including next-generation standard H.266/VVC on MS-SSIM metric. Moreover, visual analysis tasks (i.e., object detection and semantic segmentation) are conducted to verify the proposed compression method could preserve semantic fidelity besides signal-level precision.