# VideoQA-SC: Adaptive Semantic Communication for Video Question Answering

arXiv: https://arxiv.org/abs/2406.18538 · HTML: https://mlsanity.com/paper/2406.18538/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 60 references that are in the map.
- First posted on arXiv 17 May 2024.

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actions](https://mlsanity.com/paper/2105.08276/) (2021)
- [Importance Estimation for Neural Network Pruning](https://mlsanity.com/paper/1906.10771/) (2019)
- [TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering](https://mlsanity.com/paper/1704.04497/) (2017)
- [Zero-Shot Video Question Answering via Frozen Bidirectional Language Models](https://mlsanity.com/paper/2206.08155/) (2022)
- [Self-Chained Image-Language Model for Video Localization and Question Answering](https://mlsanity.com/paper/2305.06988/) (2023)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Semantic Communication Systems for Speech Transmission](https://mlsanity.com/paper/2102.12605/) (2021)
- [Video Question Answering: Datasets, Algorithms and Challenges](https://mlsanity.com/paper/2203.01225/) (2022)
- [Nonlinear Transform Source-Channel Coding for Semantic Communications](https://mlsanity.com/paper/2112.10961/) (2021)
- [Video as Conditional Graph Hierarchy for Multi-Granular Question Answering](https://mlsanity.com/paper/2112.06197/) (2021)
- [Semantics-Empowered Communication for Networked Intelligent Systems](https://mlsanity.com/paper/2007.11579/) (2020)
- [Wireless Image Retrieval at the Edge](https://mlsanity.com/paper/2007.10915/) (2020)
- [CDDM: Channel Denoising Diffusion Models for Wireless Semantic Communications](https://mlsanity.com/paper/2309.08895/) (2023)
- [Task-Oriented Communication for Multi-Device Cooperative Edge Inference](https://mlsanity.com/paper/2109.00172/) (2021)
- [Wireless Semantic Communications for Video Conferencing](https://mlsanity.com/paper/2204.07790/) (2022)
- [Wireless Deep Video Semantic Transmission](https://mlsanity.com/paper/2205.13129/) (2022)
- [DeepWiVe: Deep-Learning-Aided Wireless Video Transmission](https://mlsanity.com/paper/2111.13034/) (2021)
- [A Unified Multi-Task Semantic Communication System for Multimodal Data](https://mlsanity.com/paper/2209.07689/) (2022)
- [Deep Joint Source-Channel Coding for Semantic Communications](https://mlsanity.com/paper/2211.08747/) (2022)
- [Generative Semantic Communication: Diffusion Models Beyond Bit Recovery](https://mlsanity.com/paper/2306.04321/) (2023)
- [Deep Learning Enabled Semantic Communications with Speech Recognition and Synthesis](https://mlsanity.com/paper/2205.04603/) (2022)
- [Contrastive Video Question Answering via Video Graph Transformer](https://mlsanity.com/paper/2302.13668/) (2023)
- [Semantic-preserved Communication System for Highly Efficient Speech Transmission](https://mlsanity.com/paper/2205.12727/) (2022)
- [High Perceptual Quality Wireless Image Delivery with Denoising Diffusion Models](https://mlsanity.com/paper/2309.15889/) (2023)
- [A Robust Deep Learning Enabled Semantic Communication System for Text](https://mlsanity.com/paper/2206.02596/) (2022)

## Built on by (most cited first, 3 of 3)

- [Building Trustworthy Multimodal AI: A Review of Fairness, Transparency, and Ethics in Vision-Language Tasks](https://mlsanity.com/paper/2504.13199/) (2025)
- [Hierarchy-Aware and Channel-Adaptive Semantic Communication for Bandwidth-Limited Data Fusion](https://mlsanity.com/paper/2503.17777/) (2025)
- [ChronoSC: Task-Oriented Semantic Communication via Temporal-to-Color Encoding](https://mlsanity.com/paper/2605.16388/) (2026)

## Abstract (arXiv)

Although semantic communication (SC) has shown its potential in efficiently transmitting multimodal data such as texts, speeches and images, SC for videos has focused primarily on pixel-level reconstruction. However, these SC systems may be suboptimal for downstream intelligent tasks. Moreover, SC systems without pixel-level video reconstruction present advantages by achieving higher bandwidth efficiency and real-time performance of various intelligent tasks. The difficulty in such system design lies in the extraction of task-related compact semantic representations and their accurate delivery over noisy channels. In this paper, we propose an end-to-end SC system, named VideoQA-SC for video question answering (VideoQA) tasks. Our goal is to accomplish VideoQA tasks directly based on video semantics over noisy or fading wireless channels, bypassing the need for video reconstruction at the receiver. To this end, we develop a spatiotemporal semantic encoder for effective video semantic extraction, and a learning-based bandwidth-adaptive deep joint source-channel coding (DJSCC) scheme for efficient and robust video semantic transmission. Experiments demonstrate that VideoQA-SC outperforms traditional and advanced DJSCC-based SC systems that rely on video reconstruction at the receiver under a wide range of channel conditions and bandwidth constraints. In particular, when the signal-to-noise ratio is low, VideoQA-SC can improve the answer accuracy by 5.17% while saving almost 99.5% of the bandwidth at the same time, compared with the advanced DJSCC-based SC system. Our results show the great potential of SC system design for video applications.