# Transform-Based Feature Map Compression for CNN Inference

arXiv: https://arxiv.org/abs/2106.12850 · HTML: https://mlsanity.com/paper/2106.12850/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2021.

## Built on by (most cited first, 4 of 4)

- [TSPTQ-ViT: Two-scaled post-training quantization for vision transformer](https://mlsanity.com/paper/2305.12901/) (2023)
- [Compression-aware Projection with Greedy Dimension Reduction for Convolutional Neural Network Activations](https://mlsanity.com/paper/2110.08828/) (2021)
- [Improved Projection Learning for Lower Dimensional Feature Maps](https://mlsanity.com/paper/2210.15170/) (2022)
- [Frequency Regularization: Restricting Information Redundancy of Convolutional Neural Networks](https://mlsanity.com/paper/2304.07973/) (2023)

## Abstract (arXiv)

To achieve higher accuracy in machine learning tasks, very deep convolutional neural networks (CNNs) are designed recently. However, the large memory access of deep CNNs will lead to high power consumption. A variety of hardware-friendly compression methods have been proposed to reduce the data transfer bandwidth by exploiting the sparsity of feature maps. Most of them focus on designing a specialized encoding format to increase the compression ratio. Differently, we observe and exploit the sparsity distinction between activations in earlier and later layers to improve the compression ratio. We propose a novel hardware-friendly transform-based method named 1D-Discrete Cosine Transform on Channel dimension with Masks (DCT-CM), which intelligently combines DCT, masks, and a coding format to compress activations. The proposed algorithm achieves an average compression ratio of 2.9x (53% higher than the state-of-the-art transform-based feature map compression works) during inference on ResNet-50 with an 8-bit quantization scheme.