# Generative Video Semantic Communication via Multimodal Semantic Fusion with Large Model

arXiv: https://arxiv.org/abs/2502.13838 · HTML: https://mlsanity.com/paper/2502.13838/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 42 references that are in the map.
- First posted on arXiv 19 Feb 2025.

## Builds on (most cited first, 30 of 32)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [VideoComposer: Compositional Video Synthesis with Motion Controllability](https://mlsanity.com/paper/2306.02018/) (2023)
- [Lumiere: A Space-Time Diffusion Model for Video Generation](https://mlsanity.com/paper/2401.12945/) (2024)
- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](https://mlsanity.com/paper/1609.05473/) (2016)
- [Pixel Difference Networks for Efficient Edge Detection](https://mlsanity.com/paper/2108.07009/) (2021)
- [Towards image compression with perfect realism at ultra-low bitrates](https://mlsanity.com/paper/2310.10325/) (2023)
- [Nonlinear Transform Source-Channel Coding for Semantic Communications](https://mlsanity.com/paper/2112.10961/) (2021)
- [Text + Sketch: Image Compression at Ultra Low Rates](https://mlsanity.com/paper/2307.01944/) (2023)
- [Adversarial Feature Matching for Text Generation](https://mlsanity.com/paper/1706.03850/) (2017)
- [Generative Joint Source-Channel Coding for Semantic Image Transmission](https://mlsanity.com/paper/2211.13772/) (2022)
- [Wireless Deep Video Semantic Transmission](https://mlsanity.com/paper/2205.13129/) (2022)
- [DeepWiVe: Deep-Learning-Aided Wireless Video Transmission](https://mlsanity.com/paper/2111.13034/) (2021)
- [Generative AI-driven Semantic Communication Networks: Architecture, Technologies and Applications](https://mlsanity.com/paper/2401.00124/) (2023)
- [MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Model](https://mlsanity.com/paper/2402.16749/) (2024)
- [Latency-Aware Generative Semantic Communications with Pre-Trained Diffusion Models](https://mlsanity.com/paper/2403.17256/) (2024)
- [Transformer-Empowered 6G Intelligent Networks: From Massive MIMO Processing to Semantic Communication](https://mlsanity.com/paper/2205.03770/) (2022)
- [Diffusion-Aided Joint Source Channel Coding For High Realism Wireless Image Transmission](https://mlsanity.com/paper/2404.17736/) (2024)
- [I2VC: A Unified Framework for Intra- & Inter-frame Video Compression](https://mlsanity.com/paper/2405.14336/) (2024)
- [Language-Oriented Semantic Latent Representation for Image Transmission](https://mlsanity.com/paper/2405.09976/) (2024)
- [Evolving Semantic Communication with Generative Model](https://mlsanity.com/paper/2403.20237/) (2024)
- [Generative AI-Enhanced Multi-Modal Semantic Communication in Internet of Vehicles: System Design and Methodologies](https://mlsanity.com/paper/2409.15642/) (2024)
- [Near-Space Communications: the Last Piece of 6G Space-Air-Ground-Sea Integrated Network Puzzle](https://mlsanity.com/paper/2401.00283/) (2023)
- [Understand-Before-Talk (UBT): A Semantic Communication Approach to 6G Networks](https://mlsanity.com/paper/2210.06943/) (2022)

## Built on by (most cited first, 9 of 9)

- [Generative Diffusion Models for Wireless Networks: Fundamental, Architecture, and State-of-the-Art](https://mlsanity.com/paper/2507.16733/) (2025)
- [ToDMA: Large Model-Driven Massive Token Communications for Semantic Multiple Access](https://mlsanity.com/paper/2505.10946/) (2025)
- [Low-Bitrate Video Compression through Semantic-Conditioned Diffusion](https://mlsanity.com/paper/2512.00408/) (2025)
- [Resi-VidTok: An Efficient and Decomposed Progressive Tokenization Framework for Ultra-Low-Rate and Lightweight Video Transmission](https://mlsanity.com/paper/2510.25002/) (2025)
- [Generative AI Meets 6G and Beyond: Diffusion Models for Semantic Communications](https://mlsanity.com/paper/2511.08416/) (2025)
- [Ada-TokenCom: Rate-Adaptive Token Communications via Large-Model-Driven Token Compression and Generation](https://mlsanity.com/paper/2608.28086/) (2026)
- [A Causal Diffusion Model for Video Reconstruction from Ultra-Low-Bitrate Representations](https://mlsanity.com/paper/2602.13837/) (2026)
- [From Semantic to Token Communication: The Next Paradigm for Large-Model-Driven 6G Intelligent Connectivity](https://mlsanity.com/paper/2609.10714/) (2026)
- [Semantic-Aware Joint Source-Channel Optimization for Encoder-Agnostic Digital Video Communication](https://mlsanity.com/paper/2609.39296/) (2026)

## Abstract (arXiv)

Despite significant advancements in traditional syntactic communications based on Shannon's theory, these methods struggle to meet the requirements of 6G immersive communications, especially under challenging transmission conditions. With the development of generative artificial intelligence (GenAI), progress has been made in reconstructing videos using high-level semantic information. In this paper, we propose a scalable generative video semantic communication framework that extracts and transmits semantic information to achieve high-quality video reconstruction. Specifically, at the transmitter, description and other condition signals (e.g., first frame, sketches, etc.) are extracted from the source video, functioning as text and structural semantics, respectively. At the receiver, the diffusion-based GenAI large models are utilized to fuse the semantics of the multiple modalities for reconstructing the video. Simulation results demonstrate that, at an ultra-low channel bandwidth ratio (CBR), our scheme effectively captures semantic information to reconstruct videos aligned with human perception under different signal-to-noise ratios. Notably, the proposed ``First Frame+Desc." scheme consistently achieves CLIP score exceeding 0.92 at CBR = 0.0057 for SNR > 0 dB. This demonstrates its robust performance even under low SNR conditions.