# SAViR-T: Spatially Attentive Visual Reasoning with Transformers

arXiv: https://arxiv.org/abs/2206.09265 · HTML: https://mlsanity.com/paper/2206.09265/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 18 Jun 2022.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [One weird trick for parallelizing convolutional neural networks](https://mlsanity.com/paper/1404.5997/) (2014)
- [RAVEN: A Dataset for Relational and Analogical Visual rEasoNing](https://mlsanity.com/paper/1903.02741/) (2019)
- [End-to-End Dense Video Captioning with Masked Transformer](https://mlsanity.com/paper/1804.00819/) (2018)
- [Measuring abstract reasoning in neural networks](https://mlsanity.com/paper/1807.04225/) (2018)
- [Learning Joint Spatial-Temporal Transformations for Video Inpainting](https://mlsanity.com/paper/2007.10247/) (2020)
- [Rethinking Transformer-based Set Prediction for Object Detection](https://mlsanity.com/paper/2011.10881/) (2020)
- [Stratified Rule-Aware Network for Abstract Visual Reasoning](https://mlsanity.com/paper/2002.06838/) (2020)
- [Learning Perceptual Inference by Contrasting](https://mlsanity.com/paper/1912.00086/) (2019)
- [The Scattering Compositional Learner: Discovering Objects, Attributes, Relationships in Analogical Reasoning](https://mlsanity.com/paper/2007.04212/) (2020)
- [Scale-Localized Abstract Reasoning](https://mlsanity.com/paper/2009.09405/) (2020)
- [Abstract Diagrammatic Reasoning with Multiplex Graph Networks](https://mlsanity.com/paper/2006.11197/) (2020)
- [Abstract Reasoning with Distracting Features](https://mlsanity.com/paper/1912.00569/) (2019)
- [A Review of Emerging Research Directions in Abstract Visual Reasoning](https://mlsanity.com/paper/2202.10284/) (2022)
- [V-PROM: A Benchmark for Visual Reasoning Using Visual Progressive Matrices](https://mlsanity.com/paper/1907.12271/) (2019)
- [Effective Abstract Reasoning with Dual-Contrast Network](https://mlsanity.com/paper/2205.13720/) (2022)
- [Unsupervised Abstract Reasoning for Raven's Problem Matrices](https://mlsanity.com/paper/2109.10011/) (2021)

## Built on by (most cited first, 9 of 9)

- [Multi-Viewpoint and Multi-Evaluation with Felicitous Inductive Bias Boost Machine Abstract Reasoning Ability](https://mlsanity.com/paper/2210.14914/) (2022)
- [A Cognitively-Inspired Neural Architecture for Visual Abstract Reasoning Using Contrastive Perceptual and Conceptual Processing](https://mlsanity.com/paper/2309.10532/) (2023)
- [Learning Visual Abstract Reasoning through Dual-Stream Networks](https://mlsanity.com/paper/2411.19451/) (2024)
- [Triple-CFN: Separating Concepts and Features Enhances Machine Abstract Reasoning Ability](https://mlsanity.com/paper/2403.03190/) (2024)
- [Funny-Valen-Tine: Planning Solution Distribution Enhances Machine Abstract Reasoning Ability](https://mlsanity.com/paper/2407.02688/) (2024)
- [D4C: Improving Negative Example Quality to Enhance Machine Abstract Reasoning Ability](https://mlsanity.com/paper/2403.03452/) (2024)
- [GenVP: Generating Visual Puzzles with Contrastive Hierarchical VAEs](https://mlsanity.com/paper/2503.23598/) (2025)
- [Johnny: Structuring Representation Space to Enhance Machine Abstract Reasoning Ability](https://mlsanity.com/paper/2506.01970/) (2025)
- [DIO: Refining Mutual Information and Causal Chain to Enhance Machine Abstract Reasoning Ability](https://mlsanity.com/paper/2508.15387/) (2025)

## Abstract (arXiv)

We present a novel computational model, "SAViR-T", for the family of visual reasoning problems embodied in the Raven's Progressive Matrices (RPM). Our model considers explicit spatial semantics of visual elements within each image in the puzzle, encoded as spatio-visual tokens, and learns the intra-image as well as the inter-image token dependencies, highly relevant for the visual reasoning task. Token-wise relationship, modeled through a transformer-based SAViR-T architecture, extract group (row or column) driven representations by leveraging the group-rule coherence and use this as the inductive bias to extract the underlying rule representations in the top two row (or column) per token in the RPM. We use this relation representations to locate the correct choice image that completes the last row or column for the RPM. Extensive experiments across both synthetic RPM benchmarks, including RAVEN, I-RAVEN, RAVEN-FAIR, and PGM, and the natural image-based "V-PROM" demonstrate that SAViR-T sets a new state-of-the-art for visual reasoning, exceeding prior models' performance by a considerable margin.