# V-PROM: A Benchmark for Visual Reasoning Using Visual Progressive Matrices

arXiv: https://arxiv.org/abs/1907.12271 · HTML: https://mlsanity.com/paper/1907.12271/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jul 2019.

## Built on by (most cited first, 24 of 24)

- [KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Models](https://mlsanity.com/paper/2505.16707/) (2025)
- [A Benchmark for Compositional Visual Reasoning](https://mlsanity.com/paper/2206.05379/) (2022)
- [Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices](https://mlsanity.com/paper/2201.12382/) (2022)
- [Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactions](https://mlsanity.com/paper/2205.13803/) (2022)
- [A Review of Emerging Research Directions in Abstract Visual Reasoning](https://mlsanity.com/paper/2202.10284/) (2022)
- [Are Deep Neural Networks SMARTer than Second Graders?](https://mlsanity.com/paper/2212.09993/) (2022)
- [In-Context Analogical Reasoning with Pre-Trained Language Models](https://mlsanity.com/paper/2305.17626/) (2023)
- [Bongard-OpenWorld: Few-Shot Reasoning for Free-form Visual Concepts in the Real World](https://mlsanity.com/paper/2310.10207/) (2023)
- [RelViT: Concept-guided Vision Transformer for Visual Relational Reasoning](https://mlsanity.com/paper/2204.11167/) (2022)
- [Selective Replay Enhances Learning in Online Continual Analogical Reasoning](https://mlsanity.com/paper/2103.03987/) (2021)
- [Slot Abstractors: Toward Scalable Abstract Visual Reasoning](https://mlsanity.com/paper/2403.03458/) (2024)
- [Raven's Progressive Matrices Completion with Latent Gaussian Process Priors](https://mlsanity.com/paper/2103.12045/) (2021)
- [SAViR-T: Spatially Attentive Visual Reasoning with Transformers](https://mlsanity.com/paper/2206.09265/) (2022)
- [Bongard-RWR+: Real-World Representations of Fine-Grained Concepts in Bongard Problems](https://mlsanity.com/paper/2508.12026/) (2025)
- [A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics](https://mlsanity.com/paper/2103.01403/) (2021)
- [Imagine the Unseen World: A Benchmark for Systematic Generalization in Visual World Models](https://mlsanity.com/paper/2311.09064/) (2023)
- [KANDINSKYPatterns -- An experimental exploration environment for Pattern Analysis and Machine Intelligence](https://mlsanity.com/paper/2103.00519/) (2021)
- [The KANDY Benchmark: Incremental Neuro-Symbolic Learning and Reasoning with Kandinsky Patterns](https://mlsanity.com/paper/2402.17431/) (2024)
- [A Cognitive Paradigm Approach to Probe the Perception-Reasoning Interface in VLMs](https://mlsanity.com/paper/2501.13620/) (2025)
- [Visual Transformation Telling](https://mlsanity.com/paper/2305.01928/) (2023)
- [Emergent Communication for Rules Reasoning](https://mlsanity.com/paper/2311.04474/) (2023)
- [GlyphPattern: An Abstract Pattern Recognition Benchmark for Vision-Language Models](https://mlsanity.com/paper/2408.05894/) (2024)
- [Beyond the Linear Separability Ceiling: Aligning Representations in VLMs](https://mlsanity.com/paper/2507.07574/) (2025)
- [GEB-Bench: Abstract Structures Told in Many Voices](https://mlsanity.com/paper/2608.04111/) (2026)

## Abstract (arXiv)

One of the primary challenges faced by deep learning is the degree to which current methods exploit superficial statistics and dataset bias, rather than learning to generalise over the specific representations they have experienced. This is a critical concern because generalisation enables robust reasoning over unseen data, whereas leveraging superficial statistics is fragile to even small changes in data distribution. To illuminate the issue and drive progress towards a solution, we propose a test that explicitly evaluates abstract reasoning over visual data. We introduce a large-scale benchmark of visual questions that involve operations fundamental to many high-level vision tasks, such as comparisons of counts and logical operations on complex visual properties. The benchmark directly measures a method's ability to infer high-level relationships and to generalise them over image-based concepts. It includes multiple training/test splits that require controlled levels of generalization. We evaluate a range of deep learning architectures, and find that existing models, including those popular for vision-and-language tasks, are unable to solve seemingly-simple instances. Models using relational networks fare better but leave substantial room for improvement.