# Latency-Aware Generative Semantic Communications with Pre-Trained Diffusion Models

arXiv: https://arxiv.org/abs/2403.17256 · HTML: https://mlsanity.com/paper/2403.17256/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion for compression and wireless, one of 14 directions in Diffusion models.
- Builds on 22 of its 31 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Placed in

- Diffusion models / [Diffusion for compression and wireless](https://mlsanity.com/diffusion/diffusion-for-compression-and-wireless/)

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [Lumiere: A Space-Time Diffusion Model for Video Generation](https://mlsanity.com/paper/2401.12945/) (2024)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning](https://mlsanity.com/paper/2012.15409/) (2020)
- [Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff](https://mlsanity.com/paper/1901.07821/) (2019)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Nonlinear Transform Coding](https://mlsanity.com/paper/2007.03034/) (2020)
- [Semantic Communications for Future Internet: Fundamentals, Applications, and Challenges](https://mlsanity.com/paper/2207.00427/) (2022)
- [Generative AI for Semantic Communication: Architecture, Challenges, and Outlook](https://mlsanity.com/paper/2308.15483/) (2023)
- [Transformer-Empowered 6G Intelligent Networks: From Massive MIMO Processing to Semantic Communication](https://mlsanity.com/paper/2205.03770/) (2022)
- [Generative AI Meets Semantic Communication: Evolution and Revolution of Communication Tasks](https://mlsanity.com/paper/2401.06803/) (2024)
- [Extreme Video Compression with Pre-trained Diffusion Models](https://mlsanity.com/paper/2402.08934/) (2024)
- [A Joint Communication and Computation Design for Probabilistic Semantic Communications](https://mlsanity.com/paper/2402.16328/) (2024)

## Built on by (most cited first, 25 of 25)

- [Token Communications: A Large Model-Driven Framework for Cross-modal Context-aware Semantic Communications](https://mlsanity.com/paper/2502.12096/) (2025)
- [Semantics-Guided Diffusion for Deep Joint Source-Channel Coding in Wireless Image Transmission](https://mlsanity.com/paper/2501.01138/) (2025)
- [Latent Diffusion Model-Enabled Low-Latency Semantic Communication in the Presence of Semantic Ambiguities and Wireless Channel Noises](https://mlsanity.com/paper/2406.06644/) (2024)
- [Generative Video Semantic Communication via Multimodal Semantic Fusion with Large Model](https://mlsanity.com/paper/2502.13838/) (2025)
- [Lightweight Diffusion Models for Resource-Constrained Semantic Communication](https://mlsanity.com/paper/2410.02491/) (2024)
- [ToDMA: Large Model-Driven Massive Token Communications for Semantic Multiple Access](https://mlsanity.com/paper/2505.10946/) (2025)
- [Diffusion Models for Future Networks and Communications: A Comprehensive Survey](https://mlsanity.com/paper/2508.01586/) (2025)
- [Communicate Less, Synthesize the Rest: Latency-aware Intent-based Generative Semantic Multicasting with Diffusion Models](https://mlsanity.com/paper/2411.02334/) (2024)
- [Retrieval-augmented Generation for GenAI-enabled Semantic Communications](https://mlsanity.com/paper/2412.19494/) (2024)
- [Low-Complexity Semantic Packet Aggregation for Token Communication via Lookahead Search](https://mlsanity.com/paper/2506.19451/) (2025)
- [Video TokenCom: Textual Intent-Guided Multi-Rate Video Token Communications with UEP-Based Adaptive Source-Channel Coding](https://mlsanity.com/paper/2603.02470/) (2026)
- [Training-Free Multi-User Generative Semantic Communications via Null-Space Diffusion Sampling](https://mlsanity.com/paper/2405.09866/) (2024)
- [Promptus: Can Prompts Streaming Replace Video Streaming with Stable Diffusion](https://mlsanity.com/paper/2405.20032/) (2024)
- [Semantic-Aware Power Allocation for Generative Semantic Communications with Foundation Models](https://mlsanity.com/paper/2407.03050/) (2024)
- [Enabling Training-Free Semantic Communication Systems with Generative Diffusion Models](https://mlsanity.com/paper/2505.01209/) (2025)
- [From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications](https://mlsanity.com/paper/2505.22311/) (2025)
- [LaMI-GO: Latent Mixture Integration for Goal-Oriented Communications Achieving High Spectrum Efficiency](https://mlsanity.com/paper/2412.17839/) (2024)
- [Modeling and Performance Analysis for Semantic Communications Based on Empirical Results](https://mlsanity.com/paper/2504.21055/) (2025)
- [Ada-TokenCom: Rate-Adaptive Token Communications via Large-Model-Driven Token Compression and Generation](https://mlsanity.com/paper/2608.28086/) (2026)
- [See Where You Read with Eye Gaze Tracking and Large Language Model](https://mlsanity.com/paper/2409.19454/) (2024)
- [TOAST: Task-Oriented Adaptive Semantic Transmission over Dynamic Wireless Environments](https://mlsanity.com/paper/2506.21900/) (2025)
- [Foundation Model-Based Adaptive Semantic Image Transmission for Dynamic Wireless Environments](https://mlsanity.com/paper/2509.23590/) (2025)
- [Diffusion Offline Reinforcement Learning for Fair and Energy-Efficient UAV-Assisted Wireless Networks](https://mlsanity.com/paper/2606.16331/) (2026)
- [Demo: Real-time Generative Multicasting with On-Device Intent-aware Semantic Decomposition](https://mlsanity.com/paper/2608.14600/) (2026)
- [From Semantic to Token Communication: The Next Paradigm for Large-Model-Driven 6G Intelligent Connectivity](https://mlsanity.com/paper/2609.10714/) (2026)

## Abstract (arXiv)

Generative foundation AI models have recently shown great success in synthesizing natural signals with high perceptual quality using only textual prompts and conditioning signals to guide the generation process. This enables semantic communications at extremely low data rates in future wireless networks. In this paper, we develop a latency-aware semantic communications framework with pre-trained generative models. The transmitter performs multi-modal semantic decomposition on the input signal and transmits each semantic stream with the appropriate coding and communication schemes based on the intent. For the prompt, we adopt a re-transmission-based scheme to ensure reliable transmission, and for the other semantic modalities we use an adaptive modulation/coding scheme to achieve robustness to the changing wireless channel. Furthermore, we design a semantic and latency-aware scheme to allocate transmission power to different semantic modalities based on their importance subjected to semantic quality constraints. At the receiver, a pre-trained generative model synthesizes a high fidelity signal using the received multi-stream semantics. Simulation results demonstrate ultra-low-rate, low-latency, and channel-adaptive semantic communications.