# Toward a Human-Level Video Understanding Intelligence

arXiv: https://arxiv.org/abs/2110.04203 · HTML: https://mlsanity.com/paper/2110.04203/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 21 references that are in the map.
- First posted on arXiv 8 Oct 2021.

## Builds on (most cited first, 8 of 8)

- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [TVQA: Localized, Compositional Video Question Answering](https://mlsanity.com/paper/1809.01696/) (2018)
- [TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval](https://mlsanity.com/paper/2001.09099/) (2020)
- [Audio-Visual Scene-Aware Dialog](https://mlsanity.com/paper/1901.09107/) (2019)
- [DramaQA: Character-Centered Video Story Understanding with Hierarchical QA](https://mlsanity.com/paper/2005.03356/) (2020)
- [Constructing Hierarchical Q&A Datasets for Video Story Understanding](https://mlsanity.com/paper/1904.00623/) (2019)
- [Mounting Video Metadata on Transformer-based Language Model for Open-ended Video Question Answering](https://mlsanity.com/paper/2108.05158/) (2021)

## Abstract (arXiv)

We aim to develop an AI agent that can watch video clips and have a conversation with human about the video story. Developing video understanding intelligence is a significantly challenging task, and evaluation methods for adequately measuring and analyzing the progress of AI agent are lacking as well. In this paper, we propose the Video Turing Test to provide effective and practical assessments of video understanding intelligence as well as human-likeness evaluation of AI agents. We define a general format and procedure of the Video Turing Test and present a case study to confirm the effectiveness and usefulness of the proposed test.