# Task-Oriented Multi-User Semantic Communications

arXiv: https://arxiv.org/abs/2112.10255 · HTML: https://mlsanity.com/paper/2112.10255/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Dec 2021.

## Built on by (most cited first, 30 of 75)

- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Semantic Communications: Principles and Challenges](https://mlsanity.com/paper/2201.01389/) (2021)
- [Less Data, More Knowledge: Building Next Generation Semantic Communication Networks](https://mlsanity.com/paper/2211.14343/) (2022)
- [Robust Semantic Communications with Masked VQ-VAE Enabled Codebook](https://mlsanity.com/paper/2206.04011/) (2022)
- [Large AI Model Empowered Multimodal Semantic Communications](https://mlsanity.com/paper/2309.01249/) (2023)
- [Task-Oriented Communications for NextG: End-to-End Deep Learning and AI Security Aspects](https://mlsanity.com/paper/2212.09668/) (2022)
- [Machine Learning for Large-Scale Optimization in 6G Wireless Networks](https://mlsanity.com/paper/2301.03377/) (2023)
- [Multi-modal Deep Learning](https://mlsanity.com/paper/2403.03385/) (2024)
- [Curriculum Learning for Goal-Oriented Semantic Communications with a Common Language](https://mlsanity.com/paper/2204.10429/) (2022)
- [Task-Oriented Over-the-Air Computation for Multi-Device Edge AI](https://mlsanity.com/paper/2211.01255/) (2022)
- [Digital-SC: Digital Semantic Communication with Adaptive Network Split and Learned Non-Linear Quantization](https://mlsanity.com/paper/2305.13553/) (2023)
- [Latent Diffusion Model-Enabled Low-Latency Semantic Communication in the Presence of Semantic Ambiguities and Wireless Channel Noises](https://mlsanity.com/paper/2406.06644/) (2024)
- [Semantic Information Recovery in Wireless Networks](https://mlsanity.com/paper/2204.13366/) (2022)
- [Token Communication in the Era of Large Models: An Information Bottleneck-Based Approach](https://mlsanity.com/paper/2507.01728/) (2025)
- [Adaptive Resource Allocation for Semantic Communication Networks](https://mlsanity.com/paper/2312.01081/) (2023)
- [Applications of Generative AI (GAI) for Mobile and Wireless Networking: A Survey](https://mlsanity.com/paper/2405.20024/) (2024)
- [Semantic Successive Refinement: A Generative AI-aided Semantic Communication Framework](https://mlsanity.com/paper/2408.05112/) (2024)
- [Semantics-Empowered Communication: A Tutorial-cum-Survey](https://mlsanity.com/paper/2212.08487/) (2022)
- [Cognitive Semantic Communication Systems Driven by Knowledge Graph: Principle, Implementation, and Performance Evaluation](https://mlsanity.com/paper/2303.08546/) (2023)
- [Segment Anything Meets Semantic Communication](https://mlsanity.com/paper/2306.02094/) (2023)
- [One-to-Many Semantic Communication Systems: Design, Implementation, Performance Evaluation](https://mlsanity.com/paper/2209.09425/) (2022)
- [Adversarial Attacks and Defenses for Semantic Communication in Vehicular Metaverses](https://mlsanity.com/paper/2306.03528/) (2023)
- [Tackling Distribution Shifts in Task-Oriented Communication with Information Bottleneck](https://mlsanity.com/paper/2405.09514/) (2024)
- [Synchronous Multi-modal Semantic Communication System with Packet-level Coding](https://mlsanity.com/paper/2408.04535/) (2024)
- [Non-Orthogonal Multiple Access Enhanced Multi-User Semantic Communication](https://mlsanity.com/paper/2303.06597/) (2023)
- [How to Evaluate Semantic Communications for Images with ViTScore Metric?](https://mlsanity.com/paper/2309.04891/) (2023)
- [A Semantic-Aware Multiple Access Scheme for Distributed, Dynamic 6G-Based Applications](https://mlsanity.com/paper/2401.06308/) (2024)
- [Deep Learning in Physical Layer: Review on Data Driven End-to-End Communication Systems and their Enabling Semantic Applications](https://mlsanity.com/paper/2401.12800/) (2024)
- [Semantic Communication based on Large Language Model for Underwater Image Transmission](https://mlsanity.com/paper/2408.12616/) (2024)
- [Generative Semantic Communication for Joint Image Transmission and Segmentation](https://mlsanity.com/paper/2411.18005/) (2024)

## Abstract (arXiv)

While semantic communications have shown the potential in the case of single-modal single-users, its applications to the multi-user scenario remain limited. In this paper, we investigate deep learning (DL) based multi-user semantic communication systems for transmitting single-modal data and multimodal data, respectively. We will adopt three intelligent tasks, including, image retrieval, machine translation, and visual question answering (VQA) as the transmission goal of semantic communication systems. We will then propose a Transformer based unique framework to unify the structure of transmitters for different tasks. For the single-modal multi-user system, we will propose two Transformer based models, named, DeepSC-IR and DeepSC-MT, to perform image retrieval and machine translation, respectively. In this case, DeepSC-IR is trained to optimize the distance in embedding space between images and DeepSC-MT is trained to minimize the semantic errors by recovering the semantic meaning of sentences. For the multimodal multi-user system, we develop a Transformer enabled model, named, DeepSC-VQA, for the VQA task by extracting text-image information at the transmitters and fusing it at the receiver. In particular, a novel layer-wise Transformer is designed to help fuse multimodal data by adding connection between each of the encoder and decoder layers. Numerical results will show that the proposed models are superior to traditional communications in terms of the robustness to channels, computational complexity, transmission delay, and the task-execution performance at various task-specific metrics.