# Measuring Machine Intelligence Through Visual Question Answering

arXiv: https://arxiv.org/abs/1608.08716 · HTML: https://mlsanity.com/paper/1608.08716/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Aug 2016.

## Built on by (most cited first, 3 of 3)

- [Learning by Self-Explaining](https://mlsanity.com/paper/2309.08395/) (2023)
- [ILLUME: Rationalizing Vision-Language Models through Human Interactions](https://mlsanity.com/paper/2208.08241/) (2022)
- [The Meta-Turing Test](https://mlsanity.com/paper/2205.05268/) (2022)

## Abstract (arXiv)

As machines have become more intelligent, there has been a renewed interest in methods for measuring their intelligence. A common approach is to propose tasks for which a human excels, but one which machines find difficult. However, an ideal task should also be easy to evaluate and not be easily gameable. We begin with a case study exploring the recently popular task of image captioning and its limitations as a task for measuring machine intelligence. An alternative and more promising task is Visual Question Answering that tests a machine's ability to reason about language and vision. We describe a dataset unprecedented in size created for the task that contains over 760,000 human generated questions about images. Using around 10 million human generated answers, machines may be easily evaluated.