# Generative Semantic Communication for Joint Image Transmission and Segmentation

arXiv: https://arxiv.org/abs/2411.18005 · HTML: https://mlsanity.com/paper/2411.18005/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 27 Nov 2024.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Resnet in Resnet: Generalizing Residual Architectures](https://mlsanity.com/paper/1603.08029/) (2016)
- [DeepJSCC-f: Deep Joint Source-Channel Coding of Images with Feedback](https://mlsanity.com/paper/1911.11174/) (2019)
- [Task-Oriented Multi-User Semantic Communications](https://mlsanity.com/paper/2112.10255/) (2021)
- [WITT: A Wireless Image Transmission Transformer for Semantic Communications](https://mlsanity.com/paper/2211.00937/) (2022)
- [Generative AI Agents with Large Language Model for Satellite Networks via a Mixture of Experts Transmission](https://mlsanity.com/paper/2404.09134/) (2024)
- [Interactive AI with Retrieval-Augmented Generation for Next Generation Networking](https://mlsanity.com/paper/2401.11391/) (2024)
- [Successive Refinement of Images with Deep Joint Source-Channel Coding](https://mlsanity.com/paper/1903.06333/) (2019)
- [Knowledge Base Enabled Semantic Communication: A Generative Perspective](https://mlsanity.com/paper/2311.12443/) (2023)
- [Deep Learning-Empowered Semantic Communication Systems with a Shared Knowledge Base](https://mlsanity.com/paper/2311.02884/) (2023)
- [Generative Semantic Communication for Text-to-Speech Synthesis](https://mlsanity.com/paper/2410.03459/) (2024)

## Built on by (most cited first, 6 of 6)

- [Diffusion Models for Future Networks and Communications: A Comprehensive Survey](https://mlsanity.com/paper/2508.01586/) (2025)
- [Generative Semantic Communication: Architectures, Technologies, and Applications](https://mlsanity.com/paper/2412.08642/) (2024)
- [TONIC: Token-Centric Semantic Communication for Task-Oriented Wireless Systems](https://mlsanity.com/paper/2605.21553/) (2026)
- [Large AI Model-Enabled Generative Semantic Communications for Image Transmission](https://mlsanity.com/paper/2509.21394/) (2025)
- [Semantic-Aware Generative Image Transmission for Resource-Constrained Visual IoT Systems](https://mlsanity.com/paper/2606.28398/) (2026)
- [From Semantic to Token Communication: The Next Paradigm for Large-Model-Driven 6G Intelligent Connectivity](https://mlsanity.com/paper/2609.10714/) (2026)

## Abstract (arXiv)

Semantic communication has emerged as a promising technology for enhancing communication efficiency. However, most existing research emphasizes single-task reconstruction, neglecting model adaptability and generalization across multi-task systems. In this paper, we propose a novel generative semantic communication system that supports both image reconstruction and segmentation tasks. Our approach builds upon semantic knowledge bases (KBs) at both the transmitter and receiver, with each semantic KB comprising a source KB and a task KB. The source KB at the transmitter leverages a hierarchical Swin-Transformer, a generative AI scheme, to extract multi-level features from the input image. Concurrently, the counterpart source KB at the receiver utilizes hierarchical residual blocks to generate task-specific knowledge. Furthermore, the task KBs adopt a semantic similarity model to map different task requirements into pre-defined task instructions, thereby facilitating the feature selection of the source KBs. Additionally, we develop a unified residual block-based joint source and channel (JSCC) encoder and two task-specific JSCC decoders to achieve the two image tasks. In particular, a generative diffusion model is adopted to construct the JSCC decoder for the image reconstruction task. Experimental results show that our multi-task generative semantic communication system outperforms previous single-task communication systems in terms of peak signal-to-noise ratio and segmentation accuracy.