# Neural Reasoning, Fast and Slow, for Video Question Answering

arXiv: https://arxiv.org/abs/1907.04553 · HTML: https://mlsanity.com/paper/1907.04553/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2019.

## Built on by (most cited first, 7 of 7)

- [Zero-Shot Video Question Answering via Frozen Bidirectional Language Models](https://mlsanity.com/paper/2206.08155/) (2022)
- [Learning to Answer Visual Questions from Web Videos](https://mlsanity.com/paper/2205.05019/) (2022)
- [Hierarchical Object-oriented Spatio-Temporal Reasoning for Video Question Answering](https://mlsanity.com/paper/2106.13432/) (2021)
- [Video Question Answering with Iterative Video-Text Co-Tokenization](https://mlsanity.com/paper/2208.00934/) (2022)
- [Object-Centric Representation Learning for Video Question Answering](https://mlsanity.com/paper/2104.05166/) (2021)
- [Perceive, Query & Reason: Enhancing Video QA with Question-Guided Temporal Queries](https://mlsanity.com/paper/2412.19304/) (2024)
- [Extending Compositional Attention Networks for Social Reasoning in Videos](https://mlsanity.com/paper/2210.01191/) (2022)

## Abstract (arXiv)

What does it take to design a machine that learns to answer natural questions about a video? A Video QA system must simultaneously understand language, represent visual content over space-time, and iteratively transform these representations in response to lingual content in the query, and finally arriving at a sensible answer. While recent advances in lingual and visual question answering have enabled sophisticated representations and neural reasoning mechanisms, major challenges in Video QA remain on dynamic grounding of concepts, relations and actions to support the reasoning process. Inspired by the dual-process account of human reasoning, we design a dual process neural architecture, which is composed of a question-guided video processing module (System 1, fast and reactive) followed by a generic reasoning module (System 2, slow and deliberative). System 1 is a hierarchical model that encodes visual patterns about objects, actions and relations in space-time given the textual cues from the question. The encoded representation is a set of high-level visual features, which are then passed to System 2. Here multi-step inference follows to iteratively chain visual elements as instructed by the textual elements. The system is evaluated on the SVQA (synthetic) and TGIF-QA datasets (real), demonstrating competitive results, with a large margin in the case of multi-step reasoning.