# Wireless Deep Video Semantic Transmission

arXiv: https://arxiv.org/abs/2205.13129 · HTML: https://mlsanity.com/paper/2205.13129/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 54 references that are in the map.
- First posted on arXiv 26 May 2022.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [High-Fidelity Generative Image Compression](https://mlsanity.com/paper/2006.09965/) (2020)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [Deep Contextual Video Compression](https://mlsanity.com/paper/2109.15047/) (2021)
- [Sionna: An Open-Source Library for Next-Generation Physical Layer Research](https://mlsanity.com/paper/2203.11854/) (2022)
- [Nonlinear Transform Coding](https://mlsanity.com/paper/2007.03034/) (2020)
- [Density Modeling of Images using a Generalized Normalization Transformation](https://mlsanity.com/paper/1511.06281/) (2015)
- [DeepJSCC-f: Deep Joint Source-Channel Coding of Images with Feedback](https://mlsanity.com/paper/1911.11174/) (2019)
- [Semantic Communications: Principles and Challenges](https://mlsanity.com/paper/2201.01389/) (2021)
- [Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics](https://mlsanity.com/paper/2001.03569/) (2020)
- [Nonlinear Transform Source-Channel Coding for Semantic Communications](https://mlsanity.com/paper/2112.10961/) (2021)
- [Deep Learning for Joint Source-Channel Coding of Text](https://mlsanity.com/paper/1802.06832/) (2018)
- [HyperSeg: Patch-wise Hypernetwork for Real-time Semantic Segmentation](https://mlsanity.com/paper/2012.11582/) (2020)
- [Wireless Image Retrieval at the Edge](https://mlsanity.com/paper/2007.10915/) (2020)
- [DeepWiVe: Deep-Learning-Aided Wireless Video Transmission](https://mlsanity.com/paper/2111.13034/) (2021)
- [Bandwidth-Agile Image Transmission with Deep Joint Source-Channel Coding](https://mlsanity.com/paper/2009.12480/) (2020)
- [Efficient Nonlinear Transforms for Lossy Image Compression](https://mlsanity.com/paper/1802.00847/) (2018)

## Built on by (most cited first, 30 of 50)

- [Joint Source-Channel Coding: Fundamentals and Recent Progress in Practical Designs](https://mlsanity.com/paper/2409.17557/) (2024)
- [Machine Learning for Large-Scale Optimization in 6G Wireless Networks](https://mlsanity.com/paper/2301.03377/) (2023)
- [Semantic-Aware Spectrum Sharing in Internet of Vehicles Based on Deep Reinforcement Learning](https://mlsanity.com/paper/2406.07213/) (2024)
- [Robust Image Semantic Coding with Learnable CSI Fusion Masking over MIMO Fading Channels](https://mlsanity.com/paper/2406.07389/) (2024)
- [Generative Video Semantic Communication via Multimodal Semantic Fusion with Large Model](https://mlsanity.com/paper/2502.13838/) (2025)
- [Toward Adaptive Semantic Communications: Efficient Data Transmission via Online Learned Nonlinear Transform Source-Channel Coding](https://mlsanity.com/paper/2211.04339/) (2022)
- [A Wireless AI-Generated Content (AIGC) Provisioning Framework Empowered by Semantic Communication](https://mlsanity.com/paper/2310.17705/) (2023)
- [Importance-Aware Image Segmentation-based Semantic Communication for Autonomous Driving](https://mlsanity.com/paper/2401.10153/) (2024)
- [Agent-driven Generative Semantic Communication with Cross-Modality and Prediction](https://mlsanity.com/paper/2404.06997/) (2024)
- [Synchronous Multi-modal Semantic Communication System with Packet-level Coding](https://mlsanity.com/paper/2408.04535/) (2024)
- [Joint Semantic-Channel Coding and Modulation for Token Communications](https://mlsanity.com/paper/2511.15699/) (2025)
- [Performance Limits of a Deep Learning-Enabled Text Semantic Communication under Interference](https://mlsanity.com/paper/2302.14702/) (2023)
- [Non-Orthogonal Multiple Access Enhanced Multi-User Semantic Communication](https://mlsanity.com/paper/2303.06597/) (2023)
- [Deep Learning in Physical Layer: Review on Data Driven End-to-End Communication Systems and their Enabling Semantic Applications](https://mlsanity.com/paper/2401.12800/) (2024)
- [DM-MIMO: Diffusion Models for Robust Semantic Communications over MIMO Channels](https://mlsanity.com/paper/2407.05289/) (2024)
- [Integrating Pre-Trained Language Model with Physical Layer Communications](https://mlsanity.com/paper/2402.11656/) (2024)
- [Semantic Communications with Computer Vision Sensing for Edge Video Transmission](https://mlsanity.com/paper/2503.07252/) (2025)
- [MambaJSCC: Adaptive Deep Joint Source-Channel Coding with Generalized State Space Model](https://mlsanity.com/paper/2409.16592/) (2024)
- [VideoQA-SC: Adaptive Semantic Communication for Video Question Answering](https://mlsanity.com/paper/2406.18538/) (2024)
- [Editable-DeepSC: Reliable Cross-Modal Semantic Communications for Facial Editing](https://mlsanity.com/paper/2411.15702/) (2024)
- [Joint Source-Channel-Generation Coding: From Distortion-oriented Reconstruction to Semantic-consistent Generation](https://mlsanity.com/paper/2601.12808/) (2026)
- [Multi-User Semantic Fusion for Semantic Communications over Degraded Broadcast Channels](https://mlsanity.com/paper/2406.10556/) (2024)
- [Conquering High Packet-Loss Erasure: MoE Swin Transformer-Based Video Semantic Communication](https://mlsanity.com/paper/2508.01205/) (2025)
- [Semantics-enhanced Temporal Graph Networks for Content Popularity Prediction](https://mlsanity.com/paper/2301.12355/) (2023)
- [Alternate Learning based Sparse Semantic Communications for Visual Transmission](https://mlsanity.com/paper/2309.16681/) (2023)
- [Magmaw: Modality-Agnostic Adversarial Attacks on Machine Learning-Based Wireless Communication Systems](https://mlsanity.com/paper/2311.00207/) (2023)
- [A Relay System for Semantic Image Transmission based on Shared Feature Extraction and Hyperprior Entropy Compression](https://mlsanity.com/paper/2311.10492/) (2023)
- [RIS-Based On-the-Air Semantic Communications -- a Diffractional Deep Neural Network Approach](https://mlsanity.com/paper/2312.00535/) (2023)
- [Deep Learning in Wireless Communication Receiver: A Survey](https://mlsanity.com/paper/2501.17184/) (2025)
- [Modeling and Performance Analysis for Semantic Communications Based on Empirical Results](https://mlsanity.com/paper/2504.21055/) (2025)

## Abstract (arXiv)

In this paper, we design a new class of high-efficiency deep joint source-channel coding methods to achieve end-to-end video transmission over wireless channels. The proposed methods exploit nonlinear transform and conditional coding architecture to adaptively extract semantic features across video frames, and transmit semantic feature domain representations over wireless channels via deep joint source-channel coding. Our framework is collected under the name deep video semantic transmission (DVST). In particular, benefiting from the strong temporal prior provided by the feature domain context, the learned nonlinear transform function becomes temporally adaptive, resulting in a richer and more accurate entropy model guiding the transmission of current frame. Accordingly, a novel rate adaptive transmission mechanism is developed to customize deep joint source-channel coding for video sources. It learns to allocate the limited channel bandwidth within and among video frames to maximize the overall transmission performance. The whole DVST design is formulated as an optimization problem whose goal is to minimize the end-to-end transmission rate-distortion performance under perceptual quality metrics or machine vision task performance metrics. Across standard video source test sequences and various communication scenarios, experiments show that our DVST can generally surpass traditional wireless video coded transmission schemes. The proposed DVST framework can well support future semantic communications due to its video content-aware and machine vision task integration abilities.