# Video Question Answering on Screencast Tutorials

arXiv: https://arxiv.org/abs/2008.00544 · HTML: https://mlsanity.com/paper/2008.00544/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Aug 2020.

## Built on by (most cited first, 6 of 6)

- [EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding](https://mlsanity.com/paper/2308.09126/) (2023)
- [Video Question Answering: Datasets, Algorithms and Challenges](https://mlsanity.com/paper/2203.01225/) (2022)
- [AssistSR: Task-oriented Video Segment Retrieval for Personal AI Assistant](https://mlsanity.com/paper/2111.15050/) (2021)
- [YTCommentQA: Video Question Answerability in Instructional Videos](https://mlsanity.com/paper/2401.17343/) (2024)
- [Tutorly: Turning Programming Videos Into Apprenticeship Learning Environments with LLMs](https://mlsanity.com/paper/2405.12946/) (2024)
- [QTG-VQA: Question-Type-Guided Architectural for VideoQA Systems](https://mlsanity.com/paper/2409.09348/) (2024)

## Abstract (arXiv)

This paper presents a new video question answering task on screencast tutorials. We introduce a dataset including question, answer and context triples from the tutorial videos for a software. Unlike other video question answering works, all the answers in our dataset are grounded to the domain knowledge base. An one-shot recognition algorithm is designed to extract the visual cues, which helps enhance the performance of video question answering. We also propose several baseline neural network architectures based on various aspects of video contexts from the dataset. The experimental results demonstrate that our proposed models significantly improve the question answering performances by incorporating multi-modal contexts and domain knowledge.