# Integrating Pre-Trained Language Model with Physical Layer Communications

arXiv: https://arxiv.org/abs/2402.11656 · HTML: https://mlsanity.com/paper/2402.11656/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 18 Feb 2024.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [A Survey on Multimodal Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.12320/) (2023)
- [Language Modeling Is Compression](https://mlsanity.com/paper/2309.10668/) (2023)
- [Sionna: An Open-Source Library for Next-Generation Physical Layer Research](https://mlsanity.com/paper/2203.11854/) (2022)
- [Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach](https://mlsanity.com/paper/2102.04170/) (2021)
- [Semantic Communications: Principles and Challenges](https://mlsanity.com/paper/2201.01389/) (2021)
- [On-Device Machine Learning: An Algorithms and Learning Theory Perspective](https://mlsanity.com/paper/1911.00623/) (2019)
- [Wireless Semantic Communications for Video Conferencing](https://mlsanity.com/paper/2204.07790/) (2022)
- [Wireless Deep Video Semantic Transmission](https://mlsanity.com/paper/2205.13129/) (2022)
- [DeepWiVe: Deep-Learning-Aided Wireless Video Transmission](https://mlsanity.com/paper/2111.13034/) (2021)
- [Towards Semantic Communications: Deep Learning-Based Image Semantic Coding](https://mlsanity.com/paper/2208.04094/) (2022)
- [Deep Learning Enabled Semantic Communications with Speech Recognition and Synthesis](https://mlsanity.com/paper/2205.04603/) (2022)
- [Semantic-preserved Communication System for Highly Efficient Speech Transmission](https://mlsanity.com/paper/2205.12727/) (2022)
- [An Overview of 5G Advanced Evolution in 3GPP Release 18](https://mlsanity.com/paper/2201.01358/) (2022)
- [One-to-Many Semantic Communication Systems: Design, Implementation, Performance Evaluation](https://mlsanity.com/paper/2209.09425/) (2022)
- [Seq2Seq-SC: End-to-End Semantic Communication Systems with Pre-trained Language Model](https://mlsanity.com/paper/2210.15237/) (2022)

## Built on by (most cited first, 5 of 5)

- [WirelessLLM: Empowering Large Language Models Towards Wireless Intelligence](https://mlsanity.com/paper/2405.17053/) (2024)
- [WirelessGPT: A Generative Pre-trained Multi-task Learning Framework for Wireless Communication](https://mlsanity.com/paper/2502.06877/) (2025)
- [On Privacy, Security, and Trustworthiness in Distributed Wireless Large AI Models (WLAM)](https://mlsanity.com/paper/2412.02538/) (2024)
- [Empowering Large Language Models in Wireless Communication: A Novel Dataset and Fine-Tuning Framework](https://mlsanity.com/paper/2501.09631/) (2025)
- [Semantic Error Correction and Decoding for Short Block Codes](https://mlsanity.com/paper/2604.22269/) (2026)

## Abstract (arXiv)

The burgeoning field of on-device AI communication, where devices exchange information directly through embedded foundation models, such as language models (LMs), requires robust, efficient, and generalizable communication frameworks. However, integrating these frameworks with existing wireless systems and effectively managing noise and bit errors pose significant challenges. In this work, we introduce a practical ondevice AI communication framework, integrated with physical layer (PHY) communication functions, demonstrated through its performance on a link-level simulator. Our framework incorporates end-to-end training with channel noise to enhance resilience, incorporates vector quantized variational autoencoders (VQ-VAE) for efficient and robust communication, and utilizes pre-trained encoder-decoder transformers for improved generalization capabilities. Simulations, across various communication scenarios, reveal that our framework achieves a 50% reduction in transmission size while demonstrating substantial generalization ability and noise robustness under standardized 3GPP channel models.