# Recurrent Vision Transformer for Solving Visual Reasoning Problems

arXiv: https://arxiv.org/abs/2111.14576 · HTML: https://mlsanity.com/paper/2111.14576/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 34 references that are in the map.
- First posted on arXiv 29 Nov 2021.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Universal Transformers](https://mlsanity.com/paper/1807.03819/) (2018)
- [Neural Turing Machines](https://mlsanity.com/paper/1410.5401/) (2014)
- [General E(2)-Equivariant Steerable CNNs](https://mlsanity.com/paper/1911.08251/) (2019)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [CrossTransformers: spatially-aware few-shot transfer](https://mlsanity.com/paper/2007.11498/) (2020)
- [PonderNet: Learning to Ponder](https://mlsanity.com/paper/2107.05407/) (2021)
- [Measuring abstract reasoning in neural networks](https://mlsanity.com/paper/1807.04225/) (2018)
- [Combining EfficientNet and Vision Transformers for Video Deepfake Detection](https://mlsanity.com/paper/2107.02612/) (2021)
- [Fine-grained Visual Textual Alignment for Cross-Modal Retrieval using Transformer Encoders](https://mlsanity.com/paper/2008.05231/) (2020)
- [Transformer Reasoning Network for Image-Text Matching and Retrieval](https://mlsanity.com/paper/2004.09144/) (2020)
- [Understanding the computational demands underlying visual reasoning](https://mlsanity.com/paper/2108.03603/) (2021)
- [25 years of CNNs: Can we compare to human abstraction capabilities?](https://mlsanity.com/paper/1607.08366/) (2016)
- [Virtual to Real adaptation of Pedestrian Detectors](https://mlsanity.com/paper/2001.03032/) (2020)
- [Five Points to Check when Comparing Visual Perception in Humans and Machines](https://mlsanity.com/paper/2004.09406/) (2020)

## Built on by (most cited first, 9 of 9)

- [A Benchmark for Compositional Visual Reasoning](https://mlsanity.com/paper/2206.05379/) (2022)
- [GAMR: A Guided Attention Model for (visual) Reasoning](https://mlsanity.com/paper/2206.04928/) (2022)
- [Evaluating Compositional Scene Understanding in Multimodal Generative Models](https://mlsanity.com/paper/2503.23125/) (2025)
- [Deep Neural Networks Can Learn Generalizable Same-Different Visual Relations](https://mlsanity.com/paper/2310.09612/) (2023)
- [Take A Step Back: Rethinking the Two Stages in Visual Reasoning](https://mlsanity.com/paper/2407.19666/) (2024)
- [Investigating Single-Block Recurrence in Vision Transformers for Image Recognition](https://mlsanity.com/paper/2605.10661/) (2026)
- [Visual Reasoning in Object-Centric Deep Neural Networks: A Comparative Cognition Approach](https://mlsanity.com/paper/2402.12675/) (2024)
- [The role of object-centric representations, guided attention, and external memory on generalizing visual relations](https://mlsanity.com/paper/2304.07091/) (2023)
- [FruitEnsemble: MLLM-Guided Arbitration for Heterogeneous ensemble in Fine-Grained Fruit Recognition](https://mlsanity.com/paper/2605.20892/) (2026)

## Abstract (arXiv)

Although convolutional neural networks (CNNs) showed remarkable results in many vision tasks, they are still strained by simple yet challenging visual reasoning problems. Inspired by the recent success of the Transformer network in computer vision, in this paper, we introduce the Recurrent Vision Transformer (RViT) model. Thanks to the impact of recurrent connections and spatial attention in reasoning tasks, this network achieves competitive results on the same-different visual reasoning problems from the SVRT dataset. The weight-sharing both in spatial and depth dimensions regularizes the model, allowing it to learn using far fewer free parameters, using only 28k training samples. A comprehensive ablation study confirms the importance of a hybrid CNN + Transformer architecture and the role of the feedback connections, which iteratively refine the internal representation until a stable prediction is obtained. In the end, this study can lay the basis for a deeper understanding of the role of attention and recurrent connections for solving visual abstract reasoning tasks.