# Deep Residual Learning in the JPEG Transform Domain

arXiv: https://arxiv.org/abs/1812.11690 · HTML: https://mlsanity.com/paper/1812.11690/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2018.

## Built on by (most cited first, 30 of 44)

- [Parameter-Efficient Fine-Tuning with Discrete Fourier Transform](https://mlsanity.com/paper/2405.03003/) (2024)
- [Frequency Perception Network for Camouflaged Object Detection](https://mlsanity.com/paper/2308.08924/) (2023)
- [TCGL: Temporal Contrastive Graph for Self-supervised Video Representation Learning](https://mlsanity.com/paper/2112.03587/) (2021)
- [Privacy-Preserving Face Recognition with Learnable Privacy Budgets in Frequency Domain](https://mlsanity.com/paper/2207.07316/) (2022)
- [Learning Spatiotemporal Frequency-Transformer for Compressed Video Super-Resolution](https://mlsanity.com/paper/2208.03012/) (2022)
- [A Survey of Deep Learning Techniques for Weed Detection from Images](https://mlsanity.com/paper/2103.01415/) (2021)
- [TransFace++: Rethinking the Face Recognition Paradigm with a Focus on Accuracy, Efficiency, and Security](https://mlsanity.com/paper/2308.10133/) (2023)
- [Towards Discriminative Representation Learning for Unsupervised Person Re-identification](https://mlsanity.com/paper/2108.03439/) (2021)
- [RGB no more: Minimally-decoded JPEG Vision Transformers](https://mlsanity.com/paper/2211.16421/) (2022)
- [Continual Learning in the Frequency Domain](https://mlsanity.com/paper/2410.06645/) (2024)
- [FreqNet: A Frequency-domain Image Super-Resolution Network with Dicrete Cosine Transform](https://mlsanity.com/paper/2111.10800/) (2021)
- [Boosting Night-time Scene Parsing with Learnable Frequency](https://mlsanity.com/paper/2208.14241/) (2022)
- [Frequency Spectrum is More Effective for Multimodal Representation and Fusion: A Multimodal Spectrum Rumor Detector](https://mlsanity.com/paper/2312.11023/) (2023)
- [Latent-INR: A Flexible Framework for Implicit Representations of Videos with Discriminative Semantics](https://mlsanity.com/paper/2408.02672/) (2024)
- [Compressed Vision for Efficient Video Understanding](https://mlsanity.com/paper/2210.02995/) (2022)
- [Less is More: Accelerating Faster Neural Networks Straight from JPEG](https://mlsanity.com/paper/2104.00185/) (2021)
- [Prior-Induced Information Alignment for Image Matting](https://mlsanity.com/paper/2106.14439/) (2021)
- [Machine Perception-Driven Image Compression: A Layered Generative Approach](https://mlsanity.com/paper/2304.06896/) (2023)
- [Lite-Mind: Towards Efficient and Robust Brain Representation Network](https://mlsanity.com/paper/2312.03781/) (2023)
- [MaCP: Minimal yet Mighty Adaptation via Hierarchical Cosine Projection](https://mlsanity.com/paper/2410.09103/) (2024)
- [FreeGaze: Resource-efficient Gaze Estimation via Frequency Domain Contrastive Learning](https://mlsanity.com/paper/2209.06692/) (2022)
- [Few-shot Fine-grained Image Classification via Multi-Frequency Neighborhood and Double-cross Modulation](https://mlsanity.com/paper/2207.08547/) (2022)
- [Constrained Multiview Representation for Self-supervised Contrastive Learning](https://mlsanity.com/paper/2402.03456/) (2024)
- [Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions](https://mlsanity.com/paper/2108.13055/) (2021)
- [Learned Compression for Compressed Learning](https://mlsanity.com/paper/2412.09405/) (2024)
- [Bitstream Action Recognition is Byte Modeling](https://mlsanity.com/paper/2608.15695/) (2026)
- [Fake Visual Content Detection Using Two-Stream Convolutional Neural Networks](https://mlsanity.com/paper/2101.00676/) (2021)
- [Deep Learning Based Image Retrieval in the JPEG Compressed Domain](https://mlsanity.com/paper/2107.03648/) (2021)
- [Explicitly Increasing Input Information Density for Vision Transformers on Small Datasets](https://mlsanity.com/paper/2210.14319/) (2022)
- [Learning Spatiotemporal Frequency-Transformer for Low-Quality Video Super-Resolution](https://mlsanity.com/paper/2212.14046/) (2022)

## Abstract (arXiv)

We introduce a general method of performing Residual Network inference and learning in the JPEG transform domain that allows the network to consume compressed images as input. Our formulation leverages the linearity of the JPEG transform to redefine convolution and batch normalization with a tune-able numerical approximation for ReLu. The result is mathematically equivalent to the spatial domain network up to the ReLu approximation accuracy. A formulation for image classification and a model conversion algorithm for spatial domain networks are given as examples of the method. We show that the sparsity of the JPEG format allows for faster processing of images with little to no penalty in the network accuracy.