# Video Question Generation via Cross-Modal Self-Attention Networks Learning

arXiv: https://arxiv.org/abs/1907.03049 · HTML: https://mlsanity.com/paper/1907.03049/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2019.

## Built on by (most cited first, 4 of 4)

- [End-to-End Video Question-Answer Generation with Generator-Pretester Network](https://mlsanity.com/paper/2101.01447/) (2021)
- [Language Models are Causal Knowledge Extractors for Zero-shot Video Question Answering](https://mlsanity.com/paper/2304.03754/) (2023)
- [ECIS-VQG: Generation of Entity-centric Information-seeking Questions from Videos](https://mlsanity.com/paper/2410.09776/) (2024)
- [From text to multimodal: a survey of adversarial example generation in question answering systems](https://mlsanity.com/paper/2312.16156/) (2023)

## Abstract (arXiv)

We introduce a novel task, Video Question Generation (Video QG). A Video QG model automatically generates questions given a video clip and its corresponding dialogues. Video QG requires a range of skills -- sentence comprehension, temporal relation, the interplay between vision and language, and the ability to ask meaningful questions. To address this, we propose a novel semantic rich cross-modal self-attention (SRCMSA) network to aggregate the multi-modal and diverse features. To be more precise, we enhance the video frames semantic by integrating the object-level information, and we jointly consider the cross-modal attention for the video question generation task. Excitingly, our proposed model remarkably improves the baseline from 7.58 to 14.48 in the BLEU-4 score on the TVQA dataset. Most of all, we arguably pave a novel path toward understanding the challenging video input and we provide detailed analysis in terms of diversity, which ushers the avenues for future investigations.