# Semantic Communication with Adaptive Universal Transformer

arXiv: https://arxiv.org/abs/2108.09119 · HTML: https://mlsanity.com/paper/2108.09119/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 14 references that are in the map.
- First posted on arXiv 20 Aug 2021.

## Builds on (most cited first, 7 of 7)

- [Universal Transformers](https://mlsanity.com/paper/1807.03819/) (2018)
- [Adaptive Computation Time for Recurrent Neural Networks](https://mlsanity.com/paper/1603.08983/) (2016)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [A Lite Distributed Semantic Communication System for Internet of Things](https://mlsanity.com/paper/2007.11095/) (2020)
- [Deep Learning-Based Communication Over the Air](https://mlsanity.com/paper/1707.03384/) (2017)
- [Deep Learning for Joint Source-Channel Coding of Text](https://mlsanity.com/paper/1802.06832/) (2018)
- [DeepTurbo: Deep Turbo Decoder](https://mlsanity.com/paper/1903.02295/) (2019)

## Built on by (most cited first, 22 of 22)

- [Less Data, More Knowledge: Building Next Generation Semantic Communication Networks](https://mlsanity.com/paper/2211.14343/) (2022)
- [Task-Oriented Image Transmission for Scene Classification in Unmanned Aerial Systems](https://mlsanity.com/paper/2112.10948/) (2021)
- [Reinforcement Learning-powered Semantic Communication via Semantic Similarity](https://mlsanity.com/paper/2108.12121/) (2021)
- [Curriculum Learning for Goal-Oriented Semantic Communications with a Common Language](https://mlsanity.com/paper/2204.10429/) (2022)
- [Adaptive Semantic Token Communication for Transformer-based Edge Inference](https://mlsanity.com/paper/2505.17604/) (2025)
- [Adaptive Semantic Token Selection for AI-native Goal-oriented Communications](https://mlsanity.com/paper/2405.02330/) (2024)
- [Reasoning over the Air: A Reasoning-based Implicit Semantic-Aware Communication Framework](https://mlsanity.com/paper/2306.11229/) (2023)
- [Semantics-Empowered Communication: A Tutorial-cum-Survey](https://mlsanity.com/paper/2212.08487/) (2022)
- [Deep Learning in Physical Layer: Review on Data Driven End-to-End Communication Systems and their Enabling Semantic Applications](https://mlsanity.com/paper/2401.12800/) (2024)
- [Semantic Edge Computing and Semantic Communications in 6G Networks: A Unifying Survey and Research Challenges](https://mlsanity.com/paper/2411.18199/) (2024)
- [Towards Ubiquitous Semantic Metaverse: Challenges, Approaches, and Opportunities](https://mlsanity.com/paper/2307.06687/) (2023)
- [Knowledge Enhanced Semantic Communication Receiver](https://mlsanity.com/paper/2302.07727/) (2023)
- [Multi-User Semantic Fusion for Semantic Communications over Degraded Broadcast Channels](https://mlsanity.com/paper/2406.10556/) (2024)
- [Semantic-Native Communication: A Simplicial Complex Perspective](https://mlsanity.com/paper/2210.16970/) (2022)
- [Alternate Learning based Sparse Semantic Communications for Visual Transmission](https://mlsanity.com/paper/2309.16681/) (2023)
- [In-Context Source and Channel Coding](https://mlsanity.com/paper/2601.10267/) (2026)
- [EXK-SC: A Semantic Communication Model Based on Information Framework Expansion and Knowledge Collision](https://mlsanity.com/paper/2210.13047/) (2022)
- [Joint Semantic-Native Communication and Inference via Minimal Simplicial Structures](https://mlsanity.com/paper/2308.16789/) (2023)
- [Interplay of Semantic Communication and Knowledge Learning](https://mlsanity.com/paper/2402.03339/) (2024)
- [A Mathematical Theory for Learning Semantic Languages by Abstract Learners](https://mlsanity.com/paper/2404.07009/) (2024)
- [SemAI: Semantic Artificial Intelligence-enhanced DNA storage for Internet-of-Things](https://mlsanity.com/paper/2409.12213/) (2024)
- [Contextual Memory-Enhanced Source Coding for Low-SNR Communications](https://mlsanity.com/paper/2605.04400/) (2026)

## Abstract (arXiv)

With the development of deep learning (DL), natural language processing (NLP) makes it possible for us to analyze and understand a large amount of language texts. Accordingly, we can achieve a semantic communication in terms of joint semantic source and channel coding over a noisy channel with the help of NLP. However, the existing method to realize this goal is to use a fixed transformer of NLP while ignoring the difference of semantic information contained in each sentence. To solve this problem, we propose a new semantic communication system based on Universal Transformer. Compared with the traditional transformer, an adaptive circulation mechanism is introduced in the Universal Transformer. Through the introduction of the circulation mechanism, the new semantic communication system can be more flexible to transmit sentences with different semantic information, and achieve better end-to-end performance under various channel conditions.