# A Unified Multi-Task Semantic Communication System for Multimodal Data

arXiv: https://arxiv.org/abs/2209.07689 · HTML: https://mlsanity.com/paper/2209.07689/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2022.

## Built on by (most cited first, 30 of 41)

- [Generative Diffusion Models for Wireless Networks: Fundamental, Architecture, and State-of-the-Art](https://mlsanity.com/paper/2507.16733/) (2025)
- [Adaptive Resource Allocation for Semantic Communication Networks](https://mlsanity.com/paper/2312.01081/) (2023)
- [Applications of Generative AI (GAI) for Mobile and Wireless Networking: A Survey](https://mlsanity.com/paper/2405.20024/) (2024)
- [Semantics-Empowered Communication: A Tutorial-cum-Survey](https://mlsanity.com/paper/2212.08487/) (2022)
- [Tackling Distribution Shifts in Task-Oriented Communication with Information Bottleneck](https://mlsanity.com/paper/2405.09514/) (2024)
- [Synchronous Multi-modal Semantic Communication System with Packet-level Coding](https://mlsanity.com/paper/2408.04535/) (2024)
- [Deep Learning in Physical Layer: Review on Data Driven End-to-End Communication Systems and their Enabling Semantic Applications](https://mlsanity.com/paper/2401.12800/) (2024)
- [An Efficient Federated Learning Framework for Training Semantic Communication System](https://mlsanity.com/paper/2310.13236/) (2023)
- [Diff-GO: Diffusion Goal-Oriented Communications to Achieve Ultra-High Spectrum Efficiency](https://mlsanity.com/paper/2312.02984/) (2023)
- [Semantic-Aware Resource Management for C-V2X Platooning via Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2411.04672/) (2024)
- [Semantic Edge Computing and Semantic Communications in 6G Networks: A Unifying Survey and Research Challenges](https://mlsanity.com/paper/2411.18199/) (2024)
- [Task-Oriented Semantic Communication in Large Multimodal Models-based Vehicle Networks](https://mlsanity.com/paper/2505.02413/) (2025)
- [Deep Joint Semantic Coding and Beamforming for Near-Space Airship-Borne Massive MIMO Network](https://mlsanity.com/paper/2405.19889/) (2024)
- [Transformer-Aided Semantic Communications](https://mlsanity.com/paper/2405.01521/) (2024)
- [VideoQA-SC: Adaptive Semantic Communication for Video Question Answering](https://mlsanity.com/paper/2406.18538/) (2024)
- [Multimodal generative semantic communication based on latent diffusion model](https://mlsanity.com/paper/2408.05455/) (2024)
- [Efficient Semantic Communication Through Transformer-Aided Compression](https://mlsanity.com/paper/2412.01817/) (2024)
- [Multi-Task Semantic Communication With Graph Attention-Based Feature Correlation Extraction](https://mlsanity.com/paper/2501.02006/) (2025)
- [Multi-Task Semantic Communications via Large Models](https://mlsanity.com/paper/2503.22064/) (2025)
- [TokenCom: Vision-Language Model for Multimodal and Multitask Token Communications](https://mlsanity.com/paper/2603.00482/) (2026)
- [SC-GIR: Goal-oriented Semantic Communication via Invariant Representation Learning](https://mlsanity.com/paper/2509.01119/) (2025)
- [On Privacy, Security, and Trustworthiness in Distributed Wireless Large AI Models (WLAM)](https://mlsanity.com/paper/2412.02538/) (2024)
- [LaMI-GO: Latent Mixture Integration for Goal-Oriented Communications Achieving High Spectrum Efficiency](https://mlsanity.com/paper/2412.17839/) (2024)
- [Distributed Information Bottleneck Theory for Multi-Modal Task-Aware Semantic Communication](https://mlsanity.com/paper/2510.04000/) (2025)
- [Collaborative Edge-to-Server Inference for Vision-Language Models](https://mlsanity.com/paper/2512.16349/) (2025)
- [Adaptive Dual-Path Framework for Covert Semantic Communication](https://mlsanity.com/paper/2605.03423/) (2026)
- [Transceiver Cooperative Learning-aided Semantic Communications Against Mismatched Background Knowledge Bases](https://mlsanity.com/paper/2301.03133/) (2023)
- [Benchmarking Semantic Communications for Image Transmission Over MIMO Interference Channels](https://mlsanity.com/paper/2406.16878/) (2024)
- [Pilot-guided Multimodal Semantic Communication for Audio-Visual Event Localization](https://mlsanity.com/paper/2412.06208/) (2024)
- [Joint Transmission and Deblurring: A Semantic Communication Approach Using Events](https://mlsanity.com/paper/2501.09396/) (2025)

## Abstract (arXiv)

Task-oriented semantic communications have achieved significant performance gains. However, the employed deep neural networks in semantic communications have to be updated when the task is changed or multiple models need to be stored for performing different tasks. To address this issue, we develop a unified deep learning-enabled semantic communication system (U-DeepSC), where a unified end-to-end framework can serve many different tasks with multiple modalities of data. As the number of required features varies from task to task, we propose a vector-wise dynamic scheme that can adjust the number of transmitted symbols for different tasks. Moreover, our dynamic scheme can also adaptively adjust the number of transmitted features under different channel conditions to optimize the transmission efficiency. Particularly, we devise a lightweight feature selection module (FSM) to evaluate the importance of feature vectors, which can hierarchically drop redundant feature vectors and significantly accelerate the inference. To reduce the transmission overhead, we then design a unified codebook for feature representation to serve multiple tasks, where only the indices of these task-specific features in the codebook are transmitted. According to the simulation results, the proposed U-DeepSC achieves comparable performance to the task-oriented semantic communication system designed for a specific task but with significant reduction in both transmission overhead and model size.