# The Rate-Distortion-Perception-Classification Tradeoff: Joint Source Coding and Modulation via Inverse-Domain GANs

arXiv: https://arxiv.org/abs/2312.14792 · HTML: https://mlsanity.com/paper/2312.14792/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 22 Dec 2023.

## Builds on (most cited first, 24 of 24)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent](https://mlsanity.com/paper/1902.06720/) (2019)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [Generative Adversarial Networks for Extreme Learned Image Compression](https://mlsanity.com/paper/1804.02958/) (2018)
- [Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff](https://mlsanity.com/paper/1901.07821/) (2019)
- [Semantic Communication Systems for Speech Transmission](https://mlsanity.com/paper/2102.12605/) (2021)
- [DeepJSCC-f: Deep Joint Source-Channel Coding of Images with Feedback](https://mlsanity.com/paper/1911.11174/) (2019)
- [Deep Learning for Joint Source-Channel Coding of Text](https://mlsanity.com/paper/1802.06832/) (2018)
- [Wireless Image Retrieval at the Edge](https://mlsanity.com/paper/2007.10915/) (2020)
- [Generative Joint Source-Channel Coding for Semantic Image Transmission](https://mlsanity.com/paper/2211.13772/) (2022)
- [Bandwidth-Agile Image Transmission with Deep Joint Source-Channel Coding](https://mlsanity.com/paper/2009.12480/) (2020)
- [On Perceptual Lossy Compression: The Cost of Perceptual Reconstruction and An Optimal Training Framework](https://mlsanity.com/paper/2106.02782/) (2021)
- [Deep Joint Source Channel Coding for WirelessImage Transmission with OFDM](https://mlsanity.com/paper/2101.03909/) (2021)
- [On The Classification-Distortion-Perception Tradeoff](https://mlsanity.com/paper/1904.08816/) (2019)
- [Deep Joint Source-Channel Coding for CSI Feedback: An End-to-End Approach](https://mlsanity.com/paper/2203.16005/) (2022)
- [Cooperative Task-Oriented Communication for Multi-Modal Data with Transmission Control](https://mlsanity.com/paper/2302.02608/) (2023)

## Abstract (arXiv)

The joint source-channel coding (JSCC) framework leverages deep learning to learn from data the best codes for source and channel coding. When the output signal, rather than being binary, is directly mapped onto the IQ domain (complex-valued), we call the resulting framework joint source coding and modulation (JSCM). We consider a JSCM scenario and show the existence of a strict tradeoff between channel rate, distortion, perception, and classification accuracy, a tradeoff that we name RDPC. We then propose two image compression methods to navigate that tradeoff: the RDPCO algorithm which, under simple assumptions, directly solves the optimization problem characterizing the tradeoff, and an algorithm based on an inverse-domain generative adversarial network (ID-GAN), which is more general and achieves extreme compression. Simulation results corroborate the theoretical findings, showing that both algorithms exhibit the RDPC tradeoff. They also demonstrate that the proposed ID-GAN algorithm effectively balances image distortion, perception, and classification accuracy, and significantly outperforms traditional separation-based methods and recent deep JSCM architectures in terms of one or more of these metrics.