# Seq2Seq-SC: End-to-End Semantic Communication Systems with Pre-trained Language Model

arXiv: https://arxiv.org/abs/2210.15237 · HTML: https://mlsanity.com/paper/2210.15237/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 19 references that are in the map.
- First posted on arXiv 27 Oct 2022.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [Sionna: An Open-Source Library for Next-Generation Physical Layer Research](https://mlsanity.com/paper/2203.11854/) (2022)
- [From Semantic Communication to Semantic-aware Networking: Model, Architecture, and Open Problems](https://mlsanity.com/paper/2012.15405/) (2020)
- [Attention-based Reinforcement Learning for Real-Time UAV Semantic Communication](https://mlsanity.com/paper/2105.10716/) (2021)
- [One-to-Many Semantic Communication Systems: Design, Implementation, Performance Evaluation](https://mlsanity.com/paper/2209.09425/) (2022)

## Built on by (most cited first, 2 of 2)

- [Integrating Pre-Trained Language Model with Physical Layer Communications](https://mlsanity.com/paper/2402.11656/) (2024)
- [Task-Oriented Low-Label Semantic Communication With Self-Supervised Learning](https://mlsanity.com/paper/2505.19940/) (2025)

## Abstract (arXiv)

In this work, we propose a realistic semantic network called seq2seq-SC, designed to be compatible with 5G NR and capable of working with generalized text datasets using a pre-trained language model. The goal is to achieve unprecedented communication efficiency by focusing on the meaning of messages in semantic communication. We employ a performance metric called semantic similarity, measured by BLEU for lexical similarity and SBERT for semantic similarity. Our findings demonstrate that seq2seq-SC outperforms previous models in extracting semantically meaningful information while maintaining superior performance. This study paves the way for continued advancements in semantic communication and its prospective incorporation with future wireless systems in 6G networks.