# Large Speech Model Enabled Semantic Communication

arXiv: https://arxiv.org/abs/2512.04711 · HTML: https://mlsanity.com/paper/2512.04711/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 49 references that are in the map.
- First posted on arXiv 4 Dec 2025.

## Builds on (most cited first, 30 of 30)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [Autoregressive Image Generation using Residual Quantization](https://mlsanity.com/paper/2203.01941/) (2022)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022](https://mlsanity.com/paper/2204.02152/) (2022)
- [FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs](https://mlsanity.com/paper/2407.04051/) (2024)
- [Neural Networks Fail to Learn Periodic Functions and How to Fix It](https://mlsanity.com/paper/2006.08195/) (2020)
- [Semantic Communication Systems for Speech Transmission](https://mlsanity.com/paper/2102.12605/) (2021)
- [Semantic Communications: Principles and Challenges](https://mlsanity.com/paper/2201.01389/) (2021)
- [ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metric](https://mlsanity.com/paper/2004.09584/) (2020)
- [Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation](https://mlsanity.com/paper/2111.14826/) (2021)
- [Large AI Model-Based Semantic Communications](https://mlsanity.com/paper/2307.03492/) (2023)
- [Deep Learning Enabled Semantic Communications with Speech Recognition and Synthesis](https://mlsanity.com/paper/2205.04603/) (2022)
- [Big AI Models for 6G Wireless Networks: Opportunities, Challenges, and Research Directions](https://mlsanity.com/paper/2308.06250/) (2023)
- [Semantic-preserved Communication System for Highly Efficient Speech Transmission](https://mlsanity.com/paper/2205.12727/) (2022)
- [Joint Coding-Modulation for Digital Semantic Communications via Variational Autoencoder](https://mlsanity.com/paper/2310.06690/) (2023)
- [Language-Oriented Communication with Semantic Coding and Knowledge Distillation for Text-to-Image Generation](https://mlsanity.com/paper/2309.11127/) (2023)
- [Vector Quantized Semantic Communication System](https://mlsanity.com/paper/2209.11519/) (2022)
- [Joint Source-Channel Coding for Channel-Adaptive Digital Semantic Communications](https://mlsanity.com/paper/2311.08146/) (2023)
- [Digital-SC: Digital Semantic Communication with Adaptive Network Split and Learned Non-Linear Quantization](https://mlsanity.com/paper/2305.13553/) (2023)
- [DeepJSCC-Q: Channel Input Constrained Deep Joint Source-Channel Coding](https://mlsanity.com/paper/2111.13042/) (2021)
- [From Analog to Digital: Multi-Order Digital Joint Coding-Modulation for Semantic Communication](https://mlsanity.com/paper/2406.05437/) (2024)
- [PLCMOS -- a data-driven non-intrusive metric for the evaluation of packet loss concealment algorithms](https://mlsanity.com/paper/2305.15127/) (2023)
- [Synchronous Multi-modal Semantic Communication System with Packet-level Coding](https://mlsanity.com/paper/2408.04535/) (2024)
- [Variable Bitrate Residual Vector Quantization for Audio Coding](https://mlsanity.com/paper/2410.06016/) (2024)
- [Towards Error-Resilient Neural Speech Coding](https://mlsanity.com/paper/2207.00993/) (2022)
- [SoundSpring: Loss-Resilient Audio Transceiver with Dual-Functional Masked Language Modeling](https://mlsanity.com/paper/2501.12696/) (2025)

## Abstract (arXiv)

Existing speech semantic communication systems mainly based on Joint Source-Channel Coding (JSCC) architectures have demonstrated impressive performance, but their effectiveness remains limited by model structures specifically designed for particular tasks and datasets. Recent advances indicate that generative large models pre-trained on massive datasets, can achieve outstanding performance arexhibit exceptional performance across diverse downstream tasks with minimal fine-tuning. To exploit the rich semantic knowledge embedded in large models and enable adaptive transmission over lossy channels, we propose a Large Speech Model enabled Semantic Communication (LargeSC) system. Simultaneously achieving adaptive compression and robust transmission over lossy channels remains challenging, requiring trade-offs among compression efficiency, speech quality, and latency. In this work, we employ the Mimi as a speech codec, converting speech into discrete tokens compatible with existing network architectures. We propose an adaptive controller module that enables adaptive transmission and in-band Unequal Error Protection (UEP), dynamically adjusting to both speech content and packet loss probability under bandwidth constraints. Additionally, we employ Low-Rank Adaptation (LoRA) to finetune the Moshi foundation model for generative recovery of lost speech tokens. Simulation results show that the proposed system supports bandwidths ranging from 550 bps to 2.06 kbps, outperforms conventional baselines in speech quality under high packet loss rates and achieves an end-to-end latency of approximately 460 ms, thereby demonstrating its potential for real-time deployment.