# Large AI Model Empowered Multimodal Semantic Communications

arXiv: https://arxiv.org/abs/2309.01249 · HTML: https://mlsanity.com/paper/2309.01249/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 27 references that are in the map.
- First posted on arXiv 3 Sep 2023.

## Builds on (most cited first, 12 of 12)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [Any-to-Any Generation via Composable Diffusion](https://mlsanity.com/paper/2305.11846/) (2023)
- [Task-Oriented Multi-User Semantic Communications](https://mlsanity.com/paper/2112.10255/) (2021)
- [A Lite Distributed Semantic Communication System for Internet of Things](https://mlsanity.com/paper/2007.11095/) (2020)
- [HyperTuning: Toward Adapting Large Language Models without Back-propagation](https://mlsanity.com/paper/2211.12485/) (2022)
- [Large AI Model-Based Semantic Communications](https://mlsanity.com/paper/2307.03492/) (2023)
- [Channel Estimation for One-Bit Multiuser Massive MIMO Using Conditional GAN](https://mlsanity.com/paper/2006.11435/) (2020)

## Built on by (most cited first, 30 of 38)

- [Multi-Agent Collaboration Mechanisms: A Survey of LLMs](https://mlsanity.com/paper/2501.06322/) (2025)
- [TelecomGPT: A Framework to Build Telecom-Specfic Large Language Models](https://mlsanity.com/paper/2407.09424/) (2024)
- [AI-native Interconnect Framework for Integration of Large Language Model Technologies in 6G Systems](https://mlsanity.com/paper/2311.05842/) (2023)
- [Generative Diffusion Models for Wireless Networks: Fundamental, Architecture, and State-of-the-Art](https://mlsanity.com/paper/2507.16733/) (2025)
- [Large Generative Model Assisted 3D Semantic Communication](https://mlsanity.com/paper/2403.05783/) (2024)
- [Visual Language Model based Cross-modal Semantic Communication Systems](https://mlsanity.com/paper/2407.00020/) (2024)
- [Token Communication in the Era of Large Models: An Information Bottleneck-Based Approach](https://mlsanity.com/paper/2507.01728/) (2025)
- [Large Language Models for Wireless Communications: From Adaptation to Autonomy](https://mlsanity.com/paper/2507.21524/) (2025)
- [Generative AI Meets Semantic Communication: Evolution and Revolution of Communication Tasks](https://mlsanity.com/paper/2401.06803/) (2024)
- [ToDMA: Large Model-Driven Massive Token Communications for Semantic Multiple Access](https://mlsanity.com/paper/2505.10946/) (2025)
- [Semantic Communication based on Large Language Model for Underwater Image Transmission](https://mlsanity.com/paper/2408.12616/) (2024)
- [SIMAC: A Semantic-Driven Integrated Multimodal Sensing And Communication Framework](https://mlsanity.com/paper/2503.08726/) (2025)
- [Semantic Edge Computing and Semantic Communications in 6G Networks: A Unifying Survey and Research Challenges](https://mlsanity.com/paper/2411.18199/) (2024)
- [Semantic Communications with Computer Vision Sensing for Edge Video Transmission](https://mlsanity.com/paper/2503.07252/) (2025)
- [LAMeTA: Intent-Aware Agentic Network Optimization via a Large AI Model-Empowered Two-Stage Approach](https://mlsanity.com/paper/2505.12247/) (2025)
- [ICDM: Interference Cancellation Diffusion Models for Wireless Semantic Communications](https://mlsanity.com/paper/2505.19983/) (2025)
- [Large Generative Model-assisted Talking-face Semantic Communication System](https://mlsanity.com/paper/2411.03876/) (2024)
- [Multi-Task Semantic Communications via Large Models](https://mlsanity.com/paper/2503.22064/) (2025)
- [Multimodal LLM Integrated Semantic Communications for 6G Immersive Experiences](https://mlsanity.com/paper/2507.04621/) (2025)
- [Semantic Edge-Cloud Communication for Real-Time Urban Traffic Surveillance with ViT and LLMs over Mobile Networks](https://mlsanity.com/paper/2509.21259/) (2025)
- [Towards 6G Native-AI Edge Networks: A Semantic-Aware and Agentic Intelligence Paradigm](https://mlsanity.com/paper/2512.04405/) (2025)
- [TokenCom: Vision-Language Model for Multimodal and Multitask Token Communications](https://mlsanity.com/paper/2603.00482/) (2026)
- [Large Language Model-Driven Distributed Integrated Multimodal Sensing and Semantic Communications](https://mlsanity.com/paper/2505.18194/) (2025)
- [From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications](https://mlsanity.com/paper/2505.22311/) (2025)
- [WiFi-based Cross-Domain Gesture Recognition Using Attention Mechanism](https://mlsanity.com/paper/2512.04521/) (2025)
- [Compression Tells Intelligence: Visual Coding, Visual Token Technology, and the Unification](https://mlsanity.com/paper/2601.20742/) (2026)
- [Personalized Federated Learning for Generative AI-Assisted Semantic Communications](https://mlsanity.com/paper/2410.02450/) (2024)
- [On Privacy, Security, and Trustworthiness in Distributed Wireless Large AI Models (WLAM)](https://mlsanity.com/paper/2412.02538/) (2024)
- [LaMI-GO: Latent Mixture Integration for Goal-Oriented Communications Achieving High Spectrum Efficiency](https://mlsanity.com/paper/2412.17839/) (2024)
- [Adapting Diffusion Language Models for Lossless Pixel-Level Image Transmission](https://mlsanity.com/paper/2606.06273/) (2026)

## Abstract (arXiv)

Multimodal signals, including text, audio, image, and video, can be integrated into Semantic Communication (SC) systems to provide an immersive experience with low latency and high quality at the semantic level. However, the multimodal SC has several challenges, including data heterogeneity, semantic ambiguity, and signal distortion during transmission. Recent advancements in large AI models, particularly in the Multimodal Language Model (MLM) and Large Language Model (LLM), offer potential solutions for addressing these issues. To this end, we propose a Large AI Model-based Multimodal SC (LAM-MSC) framework, where we first present the MLM-based Multimodal Alignment (MMA) that utilizes the MLM to enable the transformation between multimodal and unimodal data while preserving semantic consistency. Then, a personalized LLM-based Knowledge Base (LKB) is proposed, which allows users to perform personalized semantic extraction or recovery through the LLM. This effectively addresses the semantic ambiguity. Finally, we apply the Conditional Generative adversarial network-based channel Estimation (CGE) for estimating the wireless channel state information. This approach effectively mitigates the impact of fading channels in SC. Finally, we conduct simulations that demonstrate the superior performance of the LAM-MSC framework.