# GenVP: Generating Visual Puzzles with Contrastive Hierarchical VAEs

arXiv: https://arxiv.org/abs/2503.23598 · HTML: https://mlsanity.com/paper/2503.23598/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 23 of its 28 references that are in the map.
- First posted on arXiv 30 Mar 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [RAVEN: A Dataset for Relational and Analogical Visual rEasoNing](https://mlsanity.com/paper/1903.02741/) (2019)
- [Measuring abstract reasoning in neural networks](https://mlsanity.com/paper/1807.04225/) (2018)
- [Stratified Rule-Aware Network for Abstract Visual Reasoning](https://mlsanity.com/paper/2002.06838/) (2020)
- [Learning Perceptual Inference by Contrasting](https://mlsanity.com/paper/1912.00086/) (2019)
- [Learning to Make Analogies by Contrasting Abstract Relational Structure](https://mlsanity.com/paper/1902.00120/) (2019)
- [The Scattering Compositional Learner: Discovering Objects, Attributes, Relationships in Analogical Reasoning](https://mlsanity.com/paper/2007.04212/) (2020)
- [Scale-Localized Abstract Reasoning](https://mlsanity.com/paper/2009.09405/) (2020)
- [Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution](https://mlsanity.com/paper/2103.14230/) (2021)
- [Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning](https://mlsanity.com/paper/2111.12990/) (2021)
- [Multi-Label Contrastive Learning for Abstract Visual Reasoning](https://mlsanity.com/paper/2012.01944/) (2020)
- [Learning to reason over visual objects](https://mlsanity.com/paper/2303.02260/) (2023)
- [Generating Correct Answers for Progressive Matrices Intelligence Tests](https://mlsanity.com/paper/2011.00496/) (2020)
- [SAViR-T: Spatially Attentive Visual Reasoning with Transformers](https://mlsanity.com/paper/2206.09265/) (2022)
- [Abstract Visual Reasoning: An Algebraic Approach for Solving Raven's Progressive Matrices](https://mlsanity.com/paper/2303.11730/) (2023)
- [A Cognitively-Inspired Neural Architecture for Visual Abstract Reasoning Using Contrastive Perceptual and Conceptual Processing](https://mlsanity.com/paper/2309.10532/) (2023)
- [Learning Visual Abstract Reasoning through Dual-Stream Networks](https://mlsanity.com/paper/2411.19451/) (2024)
- [Towards Generative Abstract Reasoning: Completing Raven's Progressive Matrix via Rule Abstraction and Selection](https://mlsanity.com/paper/2401.09966/) (2024)
- [DAReN: A Collaborative Approach Towards Reasoning And Disentangling](https://mlsanity.com/paper/2109.13156/) (2021)

## Abstract (arXiv)

Raven's Progressive Matrices (RPMs) is an established benchmark to examine the ability to perform high-level abstract visual reasoning (AVR). Despite the current success of algorithms that solve this task, humans can generalize beyond a given puzzle and create new puzzles given a set of rules, whereas machines remain locked in solving a fixed puzzle from a curated choice list. We propose Generative Visual Puzzles (GenVP), a framework to model the entire RPM generation process, a substantially more challenging task. Our model's capability spans from generating multiple solutions for one specific problem prompt to creating complete new puzzles out of the desired set of rules. Experiments on five different datasets indicate that GenVP achieves state-of-the-art (SOTA) performance both in puzzle-solving accuracy and out-of-distribution (OOD) generalization in 22 OOD scenarios. Compared to SOTA generative approaches, which struggle to solve RPMs when the feasible solution space increases, GenVP efficiently generalizes to these challenging setups. Moreover, our model demonstrates the ability to produce a wide range of complete RPMs given a set of abstract rules by effectively capturing the relationships between abstract rules and visual object properties.