# Deep Learning Enabled Semantic Communications with Speech Recognition and Synthesis

arXiv: https://arxiv.org/abs/2205.04603 · HTML: https://mlsanity.com/paper/2205.04603/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2022.

## Built on by (most cited first, 30 of 33)

- [Large AI Model-Based Semantic Communications](https://mlsanity.com/paper/2307.03492/) (2023)
- [Machine Learning for Large-Scale Optimization in 6G Wireless Networks](https://mlsanity.com/paper/2301.03377/) (2023)
- [Token Communication in the Era of Large Models: An Information Bottleneck-Based Approach](https://mlsanity.com/paper/2507.01728/) (2025)
- [Semantics-Empowered Communication: A Tutorial-cum-Survey](https://mlsanity.com/paper/2212.08487/) (2022)
- [Synchronous Multi-modal Semantic Communication System with Packet-level Coding](https://mlsanity.com/paper/2408.04535/) (2024)
- [Joint Semantic-Channel Coding and Modulation for Token Communications](https://mlsanity.com/paper/2511.15699/) (2025)
- [Deep Learning in Physical Layer: Review on Data Driven End-to-End Communication Systems and their Enabling Semantic Applications](https://mlsanity.com/paper/2401.12800/) (2024)
- [Integrating Pre-Trained Language Model with Physical Layer Communications](https://mlsanity.com/paper/2402.11656/) (2024)
- [Towards Ubiquitous Semantic Metaverse: Challenges, Approaches, and Opportunities](https://mlsanity.com/paper/2307.06687/) (2023)
- [Deep Joint Semantic Coding and Beamforming for Near-Space Airship-Borne Massive MIMO Network](https://mlsanity.com/paper/2405.19889/) (2024)
- [VideoQA-SC: Adaptive Semantic Communication for Video Question Answering](https://mlsanity.com/paper/2406.18538/) (2024)
- [Multi-Task Semantic Communications via Large Models](https://mlsanity.com/paper/2503.22064/) (2025)
- [Asymmetric Diffusion Based Channel-Adaptive Secure Wireless Semantic Communications](https://mlsanity.com/paper/2310.19439/) (2023)
- [Generative Semantic Communication for Text-to-Speech Synthesis](https://mlsanity.com/paper/2410.03459/) (2024)
- [Semantic-Native Communication: A Simplicial Complex Perspective](https://mlsanity.com/paper/2210.16970/) (2022)
- [Enhancing Semantic Communication with Deep Generative Models -- An ICASSP Special Session Overview](https://mlsanity.com/paper/2309.02478/) (2023)
- [LaMI-GO: Latent Mixture Integration for Goal-Oriented Communications Achieving High Spectrum Efficiency](https://mlsanity.com/paper/2412.17839/) (2024)
- [Task and Perception-aware Distributed Source Coding for Correlated Speech under Bandwidth-constrained Channels](https://mlsanity.com/paper/2501.17879/) (2025)
- [Distributed Information Bottleneck Theory for Multi-Modal Task-Aware Semantic Communication](https://mlsanity.com/paper/2510.04000/) (2025)
- [SemAI: Semantic Artificial Intelligence-enhanced DNA storage for Internet-of-Things](https://mlsanity.com/paper/2409.12213/) (2024)
- [Integrating Semantic Communication and Human Decision-Making into an End-to-End Sensing-Decision Framework](https://mlsanity.com/paper/2412.05103/) (2024)
- [Development of an Inclusive Educational Platform Using Open Technologies and Machine Learning: A Case Study on Accessibility Enhancement](https://mlsanity.com/paper/2503.15501/) (2025)
- [MFH: A Multi-faceted Heuristic Algorithm Selection Approach for Software Verification](https://mlsanity.com/paper/2503.22228/) (2025)
- [Task-Adaptive Semantic Communications with Controllable Diffusion-based Data Regeneration](https://mlsanity.com/paper/2505.07980/) (2025)
- [TACO: Rethinking Semantic Communications with Task Adaptation and Context Embedding](https://mlsanity.com/paper/2505.10834/) (2025)
- [Distributionally Robust Wireless Semantic Communication with Large AI Models](https://mlsanity.com/paper/2506.03167/) (2025)
- [Knowledge Graph-Based Explainable and Generalized Zero-Shot Semantic Communications](https://mlsanity.com/paper/2507.02291/) (2025)
- [User-Intent-Driven Semantic Communication via Adaptive Deep Understanding](https://mlsanity.com/paper/2508.05884/) (2025)
- [Large Speech Model Enabled Semantic Communication](https://mlsanity.com/paper/2512.04711/) (2025)
- [AudioScene: Integrating Object-Event Audio into 3D Scenes](https://mlsanity.com/paper/2512.07845/) (2025)

## Abstract (arXiv)

In this paper, we develop a deep learning based semantic communication system for speech transmission, named DeepSC-ST. We take the speech recognition and speech synthesis as the transmission tasks of the communication system, respectively. First, the speech recognition-related semantic features are extracted for transmission by a joint semantic-channel encoder and the text is recovered at the receiver based on the received semantic features, which significantly reduces the required amount of data transmission without performance degradation. Then, we perform speech synthesis at the receiver, which dedicates to re-generate the speech signals by feeding the recognized text and the speaker information into a neural network module. To enable the DeepSC-ST adaptive to dynamic channel environments, we identify a robust model to cope with different channel conditions. According to the simulation results, the proposed DeepSC-ST significantly outperforms conventional communication systems and existing DL-enabled communication systems, especially in the low signal-to-noise ratio (SNR) regime. A software demonstration is further developed as a proof-of-concept of the DeepSC-ST.