# Towards Semantic Communications: Deep Learning-Based Image Semantic Coding

arXiv: https://arxiv.org/abs/2208.04094 · HTML: https://mlsanity.com/paper/2208.04094/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 53 references that are in the map.
- First posted on arXiv 8 Aug 2022.

## Builds on (most cited first, 30 of 41)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [High-Fidelity Generative Image Compression](https://mlsanity.com/paper/2006.09965/) (2020)
- [Lossy Image Compression with Compressive Autoencoders](https://mlsanity.com/paper/1703.00395/) (2017)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks](https://mlsanity.com/paper/1908.05033/) (2019)
- [Generative Adversarial Networks for Extreme Learned Image Compression](https://mlsanity.com/paper/1804.02958/) (2018)
- [UnitBox: An Advanced Object Detection Network](https://mlsanity.com/paper/1608.01471/) (2016)
- [Full Resolution Image Compression with Recurrent Neural Networks](https://mlsanity.com/paper/1608.05148/) (2016)
- [Variable Rate Image Compression with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06085/) (2015)
- [Context-adaptive Entropy Model for End-to-end Optimized Image Compression](https://mlsanity.com/paper/1809.10452/) (2018)
- [Conditional Probability Models for Deep Image Compression](https://mlsanity.com/paper/1801.04260/) (2018)
- [Real-Time Adaptive Image Compression](https://mlsanity.com/paper/1705.05823/) (2017)
- [Understanding How Image Quality Affects Deep Neural Networks](https://mlsanity.com/paper/1604.04004/) (2016)
- [Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks](https://mlsanity.com/paper/1703.10114/) (2017)
- [Semantic Communications: Principles and Challenges](https://mlsanity.com/paper/2201.01389/) (2021)
- [A Lite Distributed Semantic Communication System for Internet of Things](https://mlsanity.com/paper/2007.11095/) (2020)
- [Deep Learning for Joint Source-Channel Coding of Text](https://mlsanity.com/paper/1802.06832/) (2018)
- [Variable-Rate Deep Image Compression through Spatially-Adaptive Feature Transform](https://mlsanity.com/paper/2108.09551/) (2021)

## Built on by (most cited first, 30 of 36)

- [High-Fidelity Image Compression with Score-based Generative Models](https://mlsanity.com/paper/2305.18231/) (2023)
- [Machine Learning for Large-Scale Optimization in 6G Wireless Networks](https://mlsanity.com/paper/2301.03377/) (2023)
- [Semantic-Aware Spectrum Sharing in Internet of Vehicles Based on Deep Reinforcement Learning](https://mlsanity.com/paper/2406.07213/) (2024)
- [Latent Diffusion Model-Enabled Low-Latency Semantic Communication in the Presence of Semantic Ambiguities and Wireless Channel Noises](https://mlsanity.com/paper/2406.06644/) (2024)
- [OFDM-Based Digital Semantic Communication with Importance Awareness](https://mlsanity.com/paper/2401.02178/) (2024)
- [Visual Language Model based Cross-modal Semantic Communication Systems](https://mlsanity.com/paper/2407.00020/) (2024)
- [Adaptive Resource Allocation for Semantic Communication Networks](https://mlsanity.com/paper/2312.01081/) (2023)
- [Semantics-Empowered Communication: A Tutorial-cum-Survey](https://mlsanity.com/paper/2212.08487/) (2022)
- [Optimization of Image Transmission in a Cooperative Semantic Communication Networks](https://mlsanity.com/paper/2301.00433/) (2023)
- [Segment Anything Meets Semantic Communication](https://mlsanity.com/paper/2306.02094/) (2023)
- [Synchronous Multi-modal Semantic Communication System with Packet-level Coding](https://mlsanity.com/paper/2408.04535/) (2024)
- [How to Evaluate Semantic Communications for Images with ViTScore Metric?](https://mlsanity.com/paper/2309.04891/) (2023)
- [Integrating Pre-Trained Language Model with Physical Layer Communications](https://mlsanity.com/paper/2402.11656/) (2024)
- [Semantic Edge Computing and Semantic Communications in 6G Networks: A Unifying Survey and Research Challenges](https://mlsanity.com/paper/2411.18199/) (2024)
- [Towards Ubiquitous Semantic Metaverse: Challenges, Approaches, and Opportunities](https://mlsanity.com/paper/2307.06687/) (2023)
- [Goal-Oriented Semantic Communication for Wireless Image Transmission via Stable Diffusion](https://mlsanity.com/paper/2408.00428/) (2024)
- [Editable-DeepSC: Reliable Cross-Modal Semantic Communications for Facial Editing](https://mlsanity.com/paper/2411.15702/) (2024)
- [Semantic Edge-Cloud Communication for Real-Time Urban Traffic Surveillance with ViT and LLMs over Mobile Networks](https://mlsanity.com/paper/2509.21259/) (2025)
- [Multi-User Semantic Fusion for Semantic Communications over Degraded Broadcast Channels](https://mlsanity.com/paper/2406.10556/) (2024)
- [Generative Semantic Communication for Text-to-Speech Synthesis](https://mlsanity.com/paper/2410.03459/) (2024)
- [Modeling and Performance Analysis for Semantic Communications Based on Empirical Results](https://mlsanity.com/paper/2504.21055/) (2025)
- [Collaborative Edge-to-Server Inference for Vision-Language Models](https://mlsanity.com/paper/2512.16349/) (2025)
- [Scenario-Adaptive MU-MIMO OFDM Semantic Communication With Asymmetric Neural Network](https://mlsanity.com/paper/2602.13557/) (2026)
- [Resource Allocation for Semantic Communication under Physical-layer Security](https://mlsanity.com/paper/2312.04155/) (2023)
- [Object-Attribute-Relation Representation Based Video Semantic Communication](https://mlsanity.com/paper/2406.10469/) (2024)
- [SCU: An Efficient Machine Unlearning Scheme for Deep Learning Enabled Semantic Communications](https://mlsanity.com/paper/2502.19785/) (2025)
- [Few-shot Semantic Encoding and Decoding for Video Surveillance](https://mlsanity.com/paper/2505.07381/) (2025)
- [TOAST: Task-Oriented Adaptive Semantic Transmission over Dynamic Wireless Environments](https://mlsanity.com/paper/2506.21900/) (2025)
- [A Survey on Semantic Communication for Vision: Categories, Frameworks, Enabling Techniques, and Applications](https://mlsanity.com/paper/2601.22202/) (2026)
- [A Causal Diffusion Model for Video Reconstruction from Ultra-Low-Bitrate Representations](https://mlsanity.com/paper/2602.13837/) (2026)

## Abstract (arXiv)

Semantic communications has received growing interest since it can remarkably reduce the amount of data to be transmitted without missing critical information. Most existing works explore the semantic encoding and transmission for text and apply techniques in Natural Language Processing (NLP) to interpret the meaning of the text. In this paper, we conceive the semantic communications for image data that is much more richer in semantics and bandwidth sensitive. We propose an reinforcement learning based adaptive semantic coding (RL-ASC) approach that encodes images beyond pixel level. Firstly, we define the semantic concept of image data that includes the category, spatial arrangement, and visual feature as the representation unit, and propose a convolutional semantic encoder to extract semantic concepts. Secondly, we propose the image reconstruction criterion that evolves from the traditional pixel similarity to semantic similarity and perceptual performance. Thirdly, we design a novel RL-based semantic bit allocation model, whose reward is the increase in rate-semantic-perceptual performance after encoding a certain semantic concept with adaptive quantization level. Thus, the task-related information is preserved and reconstructed properly while less important data is discarded. Finally, we propose the Generative Adversarial Nets (GANs) based semantic decoder that fuses both locally and globally features via an attention module. Experimental results demonstrate that the proposed RL-ASC is noise robust and could reconstruct visually pleasant and semantic consistent image, and saves times of bit cost compared to standard codecs and other deep learning-based image codecs.