# DeepWiVe: Deep-Learning-Aided Wireless Video Transmission

arXiv: https://arxiv.org/abs/2111.13034 · HTML: https://mlsanity.com/paper/2111.13034/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 66 references that are in the map.
- First posted on arXiv 25 Nov 2021.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [FVC: A New Framework towards Deep Video Compression in Feature Space](https://mlsanity.com/paper/2105.09600/) (2021)
- [Video Compression through Image Interpolation](https://mlsanity.com/paper/1804.06919/) (2018)
- [Density Modeling of Images using a Generalized Normalization Transformation](https://mlsanity.com/paper/1511.06281/) (2015)
- [Video Compression With Rate-Distortion Autoencoders](https://mlsanity.com/paper/1908.05717/) (2019)
- [DeepJSCC-f: Deep Joint Source-Channel Coding of Images with Feedback](https://mlsanity.com/paper/1911.11174/) (2019)
- [Learned Video Compression](https://mlsanity.com/paper/1811.06981/) (2018)
- [BVI-DVC: A Training Database for Deep Video Compression](https://mlsanity.com/paper/2003.13552/) (2020)
- [Bandwidth-Agile Image Transmission with Deep Joint Source-Channel Coding](https://mlsanity.com/paper/2009.12480/) (2020)
- [Deep Generative Video Compression](https://mlsanity.com/paper/1810.02845/) (2018)
- [SparseCast: Hybrid Digital-Analog Wireless Image Transmission Exploiting Frequency Domain Sparsity](https://mlsanity.com/paper/1811.10079/) (2018)

## Built on by (most cited first, 30 of 45)

- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Semantic Communications: Principles and Challenges](https://mlsanity.com/paper/2201.01389/) (2021)
- [Generative Joint Source-Channel Coding for Semantic Image Transmission](https://mlsanity.com/paper/2211.13772/) (2022)
- [Wireless Deep Video Semantic Transmission](https://mlsanity.com/paper/2205.13129/) (2022)
- [DeepJSCC-Q: Constellation Constrained Deep Joint Source-Channel Coding](https://mlsanity.com/paper/2206.08100/) (2022)
- [Deep Learning-Enabled Semantic Communication Systems with Task-Unaware Transmitter and Dynamic Data](https://mlsanity.com/paper/2205.00271/) (2022)
- [Joint Source-Channel Coding: Fundamentals and Recent Progress in Practical Designs](https://mlsanity.com/paper/2409.17557/) (2024)
- [Semantic-Aware Spectrum Sharing in Internet of Vehicles Based on Deep Reinforcement Learning](https://mlsanity.com/paper/2406.07213/) (2024)
- [CommIN: Semantic Image Communications as an Inverse Problem with INN-Guided Diffusion Models](https://mlsanity.com/paper/2310.01130/) (2023)
- [Task-Oriented Communication for Edge Video Analytics](https://mlsanity.com/paper/2211.14049/) (2022)
- [Adaptive Resource Allocation for Semantic Communication Networks](https://mlsanity.com/paper/2312.01081/) (2023)
- [Generative Video Semantic Communication via Multimodal Semantic Fusion with Large Model](https://mlsanity.com/paper/2502.13838/) (2025)
- [Toward Adaptive Semantic Communications: Efficient Data Transmission via Online Learned Nonlinear Transform Source-Channel Coding](https://mlsanity.com/paper/2211.04339/) (2022)
- [Synchronous Multi-modal Semantic Communication System with Packet-level Coding](https://mlsanity.com/paper/2408.04535/) (2024)
- [Deep Learning in Physical Layer: Review on Data Driven End-to-End Communication Systems and their Enabling Semantic Applications](https://mlsanity.com/paper/2401.12800/) (2024)
- [Integrating Pre-Trained Language Model with Physical Layer Communications](https://mlsanity.com/paper/2402.11656/) (2024)
- [SING: Semantic Image Communications using Null-Space and INN-Guided Diffusion Models](https://mlsanity.com/paper/2503.12484/) (2025)
- [Distortion Resilience for Goal-Oriented Semantic Communication](https://mlsanity.com/paper/2309.14587/) (2023)
- [Multi-level Reliability Interface for Semantic Communications over Wireless Networks](https://mlsanity.com/paper/2407.05487/) (2024)
- [Multi-user Wireless Image Semantic Transmission over MIMO Multiple Access Channels](https://mlsanity.com/paper/2504.07969/) (2025)
- [Distributed Deep Joint Source-Channel Coding with Decoder-Only Side Information](https://mlsanity.com/paper/2310.04311/) (2023)
- [VideoQA-SC: Adaptive Semantic Communication for Video Question Answering](https://mlsanity.com/paper/2406.18538/) (2024)
- [Editable-DeepSC: Reliable Cross-Modal Semantic Communications for Facial Editing](https://mlsanity.com/paper/2411.15702/) (2024)
- [Learning on Bandwidth Constrained Multi-Source Data with MIMO-inspired DPP MAP Inference](https://mlsanity.com/paper/2306.02497/) (2023)
- [Multi-User Semantic Fusion for Semantic Communications over Degraded Broadcast Channels](https://mlsanity.com/paper/2406.10556/) (2024)
- [Semantic Communication for Cooperative Perception using HARQ](https://mlsanity.com/paper/2409.09042/) (2024)
- [Conquering High Packet-Loss Erasure: MoE Swin Transformer-Based Video Semantic Communication](https://mlsanity.com/paper/2508.01205/) (2025)
- [Multi-hop Deep Joint Source-Channel Coding with Deep Hash Distillation for Semantically Aligned Image Recovery](https://mlsanity.com/paper/2510.06868/) (2025)
- [Learning to Communicate with Intent: An Introduction](https://mlsanity.com/paper/2211.09613/) (2022)
- [Task-Oriented Edge Networks: Decentralized Learning Over Wireless Fronthaul](https://mlsanity.com/paper/2312.01288/) (2023)

## Abstract (arXiv)

We present DeepWiVe, the first-ever end-to-end joint source-channel coding (JSCC) video transmission scheme that leverages the power of deep neural networks (DNNs) to directly map video signals to channel symbols, combining video compression, channel coding, and modulation steps into a single neural transform. Our DNN decoder predicts residuals without distortion feedback, which improves video quality by accounting for occlusion/disocclusion and camera movements. We simultaneously train different bandwidth allocation networks for the frames to allow variable bandwidth transmission. Then, we train a bandwidth allocation network using reinforcement learning (RL) that optimizes the allocation of limited available channel bandwidth among video frames to maximize overall visual quality. Our results show that DeepWiVe can overcome the cliff-effect, which is prevalent in conventional separation-based digital communication schemes, and achieve graceful degradation with the mismatch between the estimated and actual channel qualities. DeepWiVe outperforms H.264 video compression followed by low-density parity check (LDPC) codes in all channel conditions by up to 0.0462 on average in terms of the multi-scale structural similarity index measure (MS-SSIM), while beating H.265 + LDPC by up to 0.0058 on average. We also illustrate the importance of optimizing bandwidth allocation in JSCC video transmission by showing that our optimal bandwidth allocation policy is superior to the na"ive uniform allocation. We believe this is an important step towards fulfilling the potential of an end-to-end optimized JSCC wireless video transmission system that is superior to the current separation-based designs.