# Codebook-enabled Generative End-to-end Semantic Communication Powered by Transformer

arXiv: https://arxiv.org/abs/2402.16868 · HTML: https://mlsanity.com/paper/2402.16868/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 17 references that are in the map.
- First posted on arXiv 22 Jan 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Towards Robust Blind Face Restoration with Codebook Lookup Transformer](https://mlsanity.com/paper/2206.11253/) (2022)
- [Split Learning over Wireless Networks: Parallel Design and Resource Management](https://mlsanity.com/paper/2204.08119/) (2022)
- [Robust Semantic Communications with Masked VQ-VAE Enabled Codebook](https://mlsanity.com/paper/2206.04011/) (2022)
- [Joint Task and Data Oriented Semantic Communications: A Deep Separate Source-channel Coding Scheme](https://mlsanity.com/paper/2302.13580/) (2023)
- [Holistic Network Virtualization and Pervasive Network Intelligence for 6G](https://mlsanity.com/paper/2301.00519/) (2023)
- [Knowledge Base Enabled Semantic Communication: A Generative Perspective](https://mlsanity.com/paper/2311.12443/) (2023)

## Built on by (most cited first, 2 of 2)

- [Generative Semantic Communication for Text-to-Speech Synthesis](https://mlsanity.com/paper/2410.03459/) (2024)
- [User-Intent-Driven Semantic Communication via Adaptive Deep Understanding](https://mlsanity.com/paper/2508.05884/) (2025)

## Abstract (arXiv)

Codebook-based generative semantic communication attracts increasing attention, since only indices are required to be transmitted when the codebook is shared between transmitter and receiver. However, due to the fact that the semantic relations among code vectors are not necessarily related to the distance of the corresponding code indices, the performance of the codebook-enabled semantic communication system is susceptible to the channel noise. Thus, how to improve the system robustness against the noise requires careful design. This paper proposes a robust codebook-assisted image semantic communication system, where semantic codec and codebook are first jointly constructed, and then vector-to-index transformer is designed guided by the codebook to eliminate the effects of channel noise, and achieve image generation. Thanks to the assistance of the high-quality codebook to the Transformer, the generated images at the receiver outperform those of the compared methods in terms of visual perception. In the end, numerical results and generated images demonstrate the advantages of the generative semantic communication method over JPEG+LDPC and traditional joint source channel coding (JSCC) methods.