# Learning to reason over visual objects

arXiv: https://arxiv.org/abs/2303.02260 · HTML: https://mlsanity.com/paper/2303.02260/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 32 of its 46 references that are in the map.
- First posted on arXiv 3 Mar 2023.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Multi-Object Representation Learning with Iterative Variational Inference](https://mlsanity.com/paper/1903.00450/) (2019)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [RAVEN: A Dataset for Relational and Analogical Visual rEasoNing](https://mlsanity.com/paper/1903.02741/) (2019)
- [Measuring abstract reasoning in neural networks](https://mlsanity.com/paper/1807.04225/) (2018)
- [SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models](https://mlsanity.com/paper/2210.05861/) (2022)
- [Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs](https://mlsanity.com/paper/1901.07017/) (2019)
- [GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement](https://mlsanity.com/paper/2104.09958/) (2021)
- [Are Disentangled Representations Helpful for Abstract Visual Reasoning?](https://mlsanity.com/paper/1905.12506/) (2019)
- [Stratified Rule-Aware Network for Abstract Visual Reasoning](https://mlsanity.com/paper/2002.06838/) (2020)
- [Attention over learned object embeddings enables complex visual reasoning](https://mlsanity.com/paper/2012.08508/) (2020)
- [Learning Perceptual Inference by Contrasting](https://mlsanity.com/paper/1912.00086/) (2019)
- [Learning to Make Analogies by Contrasting Abstract Relational Structure](https://mlsanity.com/paper/1902.00120/) (2019)
- [Emergent Symbols through Binding in External Memory](https://mlsanity.com/paper/2012.14601/) (2020)
- [The Scattering Compositional Learner: Discovering Objects, Attributes, Relationships in Analogical Reasoning](https://mlsanity.com/paper/2007.04212/) (2020)
- [Scale-Localized Abstract Reasoning](https://mlsanity.com/paper/2009.09405/) (2020)
- [Abstract Diagrammatic Reasoning with Multiplex Graph Networks](https://mlsanity.com/paper/2006.11197/) (2020)
- [Learning Representations that Support Extrapolation](https://mlsanity.com/paper/2007.05059/) (2020)
- [Abstract Reasoning with Distracting Features](https://mlsanity.com/paper/1912.00569/) (2019)
- [Improving Generalization for Abstract Reasoning Tasks Using Disentangled Feature Representations](https://mlsanity.com/paper/1811.04784/) (2018)
- [Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning](https://mlsanity.com/paper/2111.12990/) (2021)
- [On Neural Architecture Inductive Biases for Relational Tasks](https://mlsanity.com/paper/2206.05056/) (2022)
- [Effective Abstract Reasoning with Dual-Contrast Network](https://mlsanity.com/paper/2205.13720/) (2022)

## Built on by (most cited first, 19 of 19)

- [Understanding the Limits of Vision Language Models Through the Lens of the Binding Problem](https://mlsanity.com/paper/2411.00238/) (2024)
- [The Relational Bottleneck as an Inductive Bias for Efficient Abstraction](https://mlsanity.com/paper/2309.06629/) (2023)
- [Neural-Logic Human-Object Interaction Detection](https://mlsanity.com/paper/2311.09817/) (2023)
- [Systematic Visual Reasoning through Object-Centric Relational Abstraction](https://mlsanity.com/paper/2306.02500/) (2023)
- [Look, Remember and Reason: Grounded reasoning in videos with language models](https://mlsanity.com/paper/2306.17778/) (2023)
- [Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Models](https://mlsanity.com/paper/2407.15589/) (2024)
- [Abstractors and relational cross-attention: An inductive bias for explicit relational reasoning in Transformers](https://mlsanity.com/paper/2304.00195/) (2023)
- [Slot Abstractors: Toward Scalable Abstract Visual Reasoning](https://mlsanity.com/paper/2403.03458/) (2024)
- [One Self-Configurable Model to Solve Many Abstract Visual Reasoning Problems](https://mlsanity.com/paper/2312.09997/) (2023)
- [Evaluating Compositional Scene Understanding in Multimodal Generative Models](https://mlsanity.com/paper/2503.23125/) (2025)
- [A-I-RAVEN and I-RAVEN-Mesh: Two New Benchmarks for Abstract Visual Reasoning](https://mlsanity.com/paper/2406.11061/) (2024)
- [Learning Visual Abstract Reasoning through Dual-Stream Networks](https://mlsanity.com/paper/2411.19451/) (2024)
- [Learning to Reason Iteratively and Parallelly for Complex Visual Reasoning Scenarios](https://mlsanity.com/paper/2411.13754/) (2024)
- [A Relational Inductive Bias for Dimensional Abstraction in Neural Networks](https://mlsanity.com/paper/2402.18426/) (2024)
- [Systematic Abductive Reasoning via Diverse Relation Representations in Vector-symbolic Architecture](https://mlsanity.com/paper/2501.11896/) (2025)
- [GenVP: Generating Visual Puzzles with Contrastive Hierarchical VAEs](https://mlsanity.com/paper/2503.23598/) (2025)
- [Advancing Generalization Across a Variety of Abstract Visual Reasoning Tasks](https://mlsanity.com/paper/2505.13391/) (2025)
- [Beyond Task-Specific Reasoning: A Unified Conditional Generative Framework for Abstract Visual Reasoning](https://mlsanity.com/paper/2507.11761/) (2025)
- [DIRCR: Dual-Inference Rule-Contrastive Reasoning for Solving RAVENs](https://mlsanity.com/paper/2604.17584/) (2026)

## Abstract (arXiv)

A core component of human intelligence is the ability to identify abstract patterns inherent in complex, high-dimensional perceptual data, as exemplified by visual reasoning tasks such as Raven's Progressive Matrices (RPM). Motivated by the goal of designing AI systems with this capacity, recent work has focused on evaluating whether neural networks can learn to solve RPM-like problems. Previous work has generally found that strong performance on these problems requires the incorporation of inductive biases that are specific to the RPM problem format, raising the question of whether such models might be more broadly useful. Here, we investigated the extent to which a general-purpose mechanism for processing visual scenes in terms of objects might help promote abstract visual reasoning. We found that a simple model, consisting only of an object-centric encoder and a transformer reasoning module, achieved state-of-the-art results on both of two challenging RPM-like benchmarks (PGM and I-RAVEN), as well as a novel benchmark with greater visual complexity (CLEVR-Matrices). These results suggest that an inductive bias for object-centric processing may be a key component of abstract visual reasoning, obviating the need for problem-specific inductive biases.