# Frequency-Aware Re-Parameterization for Over-Fitting Based Image Compression

arXiv: https://arxiv.org/abs/2310.08068 · HTML: https://mlsanity.com/paper/2310.08068/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 22 references that are in the map.
- First posted on arXiv 12 Oct 2023.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks](https://mlsanity.com/paper/1901.06523/) (2019)
- [Image and Video Compression with Neural Networks: A Review](https://mlsanity.com/paper/1904.03567/) (2019)
- [Overfitting for Fun and Profit: Instance-Adaptive Data Compression](https://mlsanity.com/paper/2101.08687/) (2021)
- [DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks](https://mlsanity.com/paper/1907.11900/) (2019)
- [Overfitting the Data: Compact Neural Video Delivery via Content-aware Feature Modulation](https://mlsanity.com/paper/2108.08202/) (2021)
- [Efficient Adaptation of Neural Network Filter for Video Compression](https://mlsanity.com/paper/2007.14267/) (2020)
- [Compressing Weight-updates for Image Artifacts Removal Neural Networks](https://mlsanity.com/paper/1905.04079/) (2019)
- [Tensor Reordering for CNN Compression](https://mlsanity.com/paper/2010.12110/) (2020)

## Abstract (arXiv)

Over-fitting-based image compression requires weights compactness for compression and fast convergence for practical use, posing challenges for deep convolutional neural networks (CNNs) based methods. This paper presents a simple re-parameterization method to train CNNs with reduced weights storage and accelerated convergence. The convolution kernels are re-parameterized as a weighted sum of discrete cosine transform (DCT) kernels enabling direct optimization in the frequency domain. Combined with L1 regularization, the proposed method surpasses vanilla convolutions by achieving a significantly improved rate-distortion with low computational cost. The proposed method is verified with extensive experiments of over-fitting-based image restoration on various datasets, achieving up to -46.12% BD-rate on top of HEIF with only 200 iterations.