# Less is More: Accelerating Faster Neural Networks Straight from JPEG

arXiv: https://arxiv.org/abs/2104.00185 · HTML: https://mlsanity.com/paper/2104.00185/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 17 references that are in the map.
- First posted on arXiv 1 Apr 2021.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [Learning Filter Basis for Convolutional Neural Network Compression](https://mlsanity.com/paper/1908.08932/) (2019)
- [Deep Residual Learning in the JPEG Transform Domain](https://mlsanity.com/paper/1812.11690/) (2018)
- [Fast object detection in compressed JPEG Images](https://mlsanity.com/paper/1904.08408/) (2019)
- [Faster and Accurate Compressed Video Action Recognition Straight from the Frequency Domain](https://mlsanity.com/paper/2012.13726/) (2020)

## Built on by (most cited first, 7 of 7)

- [DuetFace: Collaborative Privacy-Preserving Face Recognition via Channel Splitting in the Frequency Domain](https://mlsanity.com/paper/2207.07340/) (2022)
- [RGB no more: Minimally-decoded JPEG Vision Transformers](https://mlsanity.com/paper/2211.16421/) (2022)
- [A Hybrid Quantum-Classical Approach based on the Hadamard Transform for the Convolutional Layer](https://mlsanity.com/paper/2305.17510/) (2023)
- [Machine Perception-Driven Image Compression: A Layered Generative Approach](https://mlsanity.com/paper/2304.06896/) (2023)
- [DCT Perceptron Layer: A Transform Domain Approach for Convolution Layer](https://mlsanity.com/paper/2211.08577/) (2022)
- [Multichannel Orthogonal Transform-Based Perceptron Layers for Efficient ResNets](https://mlsanity.com/paper/2303.06797/) (2023)
- [CNNs for JPEGs: A Study in Computational Cost](https://mlsanity.com/paper/2309.11417/) (2023)

## Abstract (arXiv)

Most image data available are often stored in a compressed format, from which JPEG is the most widespread. To feed this data on a convolutional neural network (CNN), a preliminary decoding process is required to obtain RGB pixels, demanding a high computational load and memory usage. For this reason, the design of CNNs for processing JPEG compressed data has gained attention in recent years. In most existing works, typical CNN architectures are adapted to facilitate the learning with the DCT coefficients rather than RGB pixels. Although they are effective, their architectural changes either raise the computational costs or neglect relevant information from DCT inputs. In this paper, we examine different ways of speeding up CNNs designed for DCT inputs, exploiting learning strategies to reduce the computational complexity by taking full advantage of DCT inputs. Our experiments were conducted on the ImageNet dataset. Results show that learning how to combine all DCT inputs in a data-driven fashion is better than discarding them by hand, and its combination with a reduction of layers has proven to be effective for reducing the computational costs while retaining accuracy.