# Transferring Domain-Agnostic Knowledge in Video Question Answering

arXiv: https://arxiv.org/abs/2110.13395 · HTML: https://mlsanity.com/paper/2110.13395/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 34 references that are in the map.
- First posted on arXiv 26 Oct 2021.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge](https://mlsanity.com/paper/1906.00067/) (2019)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [TVQA: Localized, Compositional Video Question Answering](https://mlsanity.com/paper/1809.01696/) (2018)
- [Just Ask: Learning to Answer Questions from Millions of Narrated Videos](https://mlsanity.com/paper/2012.00451/) (2020)
- [FVQA: Fact-based Visual Question Answering](https://mlsanity.com/paper/1606.05433/) (2016)
- [Explicit Knowledge-based Reasoning for Visual Question Answering](https://mlsanity.com/paper/1511.02570/) (2015)
- [Location-aware Graph Convolutional Networks for Video Question Answering](https://mlsanity.com/paper/2008.09105/) (2020)
- [KnowIT VQA: Answering Knowledge-Based Questions about Videos](https://mlsanity.com/paper/1910.10706/) (2019)
- [Ask Me Anything: Free-form Visual Question Answering Based on Knowledge from External Sources](https://mlsanity.com/paper/1511.06973/) (2015)
- [Video Question Answering via Attribute-Augmented Attention Network Learning](https://mlsanity.com/paper/1707.06355/) (2017)
- [iPerceive: Applying Common-Sense Reasoning to Multi-Modal Dense Video Captioning and Video Question Answering](https://mlsanity.com/paper/2011.07735/) (2020)
- [Cross-Dataset Adaptation for Visual Question Answering](https://mlsanity.com/paper/1806.03726/) (2018)
- [Knowledge-Based Video Question Answering with Unsupervised Scene Descriptions](https://mlsanity.com/paper/2007.08751/) (2020)
- [Open-Ended Visual Question Answering by Multi-Modal Domain Adaptation](https://mlsanity.com/paper/1911.04058/) (2019)
- [Learning Answer Embeddings for Visual Question Answering](https://mlsanity.com/paper/1806.03724/) (2018)
- [On the hidden treasure of dialog in video question answering](https://mlsanity.com/paper/2103.14517/) (2021)
- [Transfer Learning via Unsupervised Task Discovery for Visual Question Answering](https://mlsanity.com/paper/1810.02358/) (2018)

## Built on by (most cited first, 7 of 7)

- [EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding](https://mlsanity.com/paper/2308.09126/) (2023)
- [Video Question Answering: Datasets, Algorithms and Challenges](https://mlsanity.com/paper/2203.01225/) (2022)
- [Knowledge Graph-extended Retrieval Augmented Generation for Question Answering](https://mlsanity.com/paper/2504.08893/) (2025)
- [A Lightweight Moment Retrieval System with Global Re-Ranking and Robust Adaptive Bidirectional Temporal Search](https://mlsanity.com/paper/2504.09298/) (2025)
- [Learning More May Not Be Better: Knowledge Transferability in Vision and Language Tasks](https://mlsanity.com/paper/2208.10758/) (2022)
- [QTG-VQA: Question-Type-Guided Architectural for VideoQA Systems](https://mlsanity.com/paper/2409.09348/) (2024)
- [A Framework for Generating Artificial Datasets to Validate Absolute and Relative Position Concepts](https://mlsanity.com/paper/2509.18177/) (2025)

## Abstract (arXiv)

Video question answering (VideoQA) is designed to answer a given question based on a relevant video clip. The current available large-scale datasets have made it possible to formulate VideoQA as the joint understanding of visual and language information. However, this training procedure is costly and still less competent with human performance. In this paper, we investigate a transfer learning method by the introduction of domain-agnostic knowledge and domain-specific knowledge. First, we develop a novel transfer learning framework, which finetunes the pre-trained model by applying domain-agnostic knowledge as the medium. Second, we construct a new VideoQA dataset with 21,412 human-generated question-answer samples for comparable transfer of knowledge. Our experiments show that: (i) domain-agnostic knowledge is transferable and (ii) our proposed transfer learning framework can boost VideoQA performance effectively.