LoopViT: Scaling Visual ARC with Looped Transformers
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Recent advances in visual reasoning have leveraged vision transformers to tackle the ARC-AGI benchmark. However, we argue that the feed-forward architecture, where computational depth is strictly bound to parameter size, falls short of capturing the iterative, algorithmic nature of human induction. In this work, we propose a recursive architecture called Loop-ViT, which decouples reasoning depth from model capacity through weight-tied recurrence. Loop-ViT iterates a weight-tied Hybrid Block, combining local convolutions and global attention, to form a latent chain of thought. Crucially, we introduce a parameter-free Dynamic Exit mechanism based on predictive entropy: the model halts inference when its internal state ``crystallizes" into a low-uncertainty attractor. Empirical results on the ARC-AGI-1 benchmark validate this perspective: our 18M model achieves 65.8% accuracy, outperforming massive 73M-parameter ensembles. These findings demonstrate that adaptive iterative computation offers a far more efficient scaling axis for visual reasoning than simply increasing network width. The code is available at https://github.com/WenjieShu/LoopViT.
Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 41 references that are in the map. First posted on arXiv 2 Feb 2026.
Builds on
28 of 41 references.
Closest ancestors · 2
Same direction, newest first
- 2024ARC Prize 2024: Technical Report5 Dec
- 2024Combining Induction and Transduction for Abstract Reasoning4 Nov
Other references · 26
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
- 2019Root Mean Square Layer Normalizationcited by 1,425
- 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
- 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
- 2024Mixture-of-Depths: Dynamically allocating compute in transformer-based language modelscited by 242
- 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 223
- 2025Hierarchical Reasoning Modelcited by 167
14 more
- 2025Scaling Latent Reasoning via Looped Language Modelscited by 145
- 2023Looped Transformers are Better at Learning Learning Algorithmscited by 135
- 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 119
- 2021Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networkscited by 108
- 2023The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domaincited by 97
- 2024Looped Transformers for Length Generalizationcited by 91
- 2024Code Repair with LLMs gives an Exploration-Exploitation Tradeoffcited by 50
- 2025PonderLM: Pretraining Language Models to Ponder in Continuous Spacecited by 44
- 2021Fast and flexible: Human program induction in abstract reasoning taskscited by 33
- 2024H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmarkcited by 32
- 2024Addressing the Abstraction and Reasoning Corpus via Procedural Example Generationcited by 31
- 2023LGViT: Dynamic Early Exiting for Accelerating Vision Transformercited by 30
- 2025ARC Is a Vision Problem!cited by 23
- 2025Understanding and Benchmarking Artificial Intelligence: OpenAI's o3 Is Not AGIcited by 19
Built on by
Citing papers per year: 2026: 14.
14 papers.
Sort:
By area:
- 2026Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformerscited by 21
- 2026Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoningcited by 12
- 2026ELT: Elastic Looped Transformers for Visual Generationcited by 11
- 2026LT2: Linear-Time Looped Transformerscited by 5
- 2026Loop the Loopies!cited by 4
- 2026Depth-adaptive Inference of Looped Language Models via Continuous Depth Batchingcited by 4
- 2026Investigating Single-Block Recurrence in Vision Transformers for Image Recognitioncited by 3
- 2026Slots, Transitions, Loops: Learning Composable World Models for ARCcited by 2
- 2026TraceViT: Grounded Trace Supervision for Visual Abstract Reasoningcited by 1
- 2026Language-Guided Abstraction for Visual Reasoningcited by 0
- 2026DiARC: Distinguishing Positive and Negative Samples Helps Improving ARC-like Reasoning Ability of Large Language Modelscited by 0
- 2026Few-Shot Demonstrations Elicit the Use of In-Context World Representations in LLMscited by 0
2 more
- 2026Reasoning with Neural Cellular Automatacited by 0
- 2026LoopVL: Recurrent Visual Intelligencecited by 0
- 2026LoopVL: Recurrent Visual Intelligence29 Sep 2026
- 2026Reasoning with Neural Cellular Automata28 Sep 2026
- 2026Few-Shot Demonstrations Elicit the Use of In-Context World Representations in LLMs21 Sep 2026
- 2026Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching10 Aug 2026
- 2026TraceViT: Grounded Trace Supervision for Visual Abstract Reasoning31 Jul 2026
- 2026Loop the Loopies!17 Jul 2026
- 2026DiARC: Distinguishing Positive and Negative Samples Helps Improving ARC-like Reasoning Ability of Large Language Models25 Jun 2026
- 2026Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers16 Jun 2026
- 2026Language-Guided Abstraction for Visual Reasoning11 Jun 2026
- 2026Slots, Transitions, Loops: Learning Composable World Models for ARC10 Jun 2026
- 2026Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning20 May 2026
- 2026LT2: Linear-Time Looped Transformers20 May 2026
2 more
- 2026Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoningcited by 12
- 2026Language-Guided Abstraction for Visual Reasoningcited by 0
- 2026DiARC: Distinguishing Positive and Negative Samples Helps Improving ARC-like Reasoning Ability of Large Language Modelscited by 0
- 2026Reasoning with Neural Cellular Automatacited by 0
- 2026Reasoning with Neural Cellular Automata28 Sep 2026
- 2026DiARC: Distinguishing Positive and Negative Samples Helps Improving ARC-like Reasoning Ability of Large Language Models25 Jun 2026
- 2026Language-Guided Abstraction for Visual Reasoning11 Jun 2026
- 2026Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning20 May 2026
- 2026Loop the Loopies!cited by 4
- 2026Loop the Loopies!17 Jul 2026
- 2026Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformerscited by 21
- 2026ELT: Elastic Looped Transformers for Visual Generationcited by 11
- 2026LT2: Linear-Time Looped Transformerscited by 5
- 2026Depth-adaptive Inference of Looped Language Models via Continuous Depth Batchingcited by 4
- 2026Investigating Single-Block Recurrence in Vision Transformers for Image Recognitioncited by 3
- 2026TraceViT: Grounded Trace Supervision for Visual Abstract Reasoningcited by 1
- 2026Few-Shot Demonstrations Elicit the Use of In-Context World Representations in LLMscited by 0
- 2026LoopVL: Recurrent Visual Intelligencecited by 0
- 2026LoopVL: Recurrent Visual Intelligence29 Sep 2026
- 2026Few-Shot Demonstrations Elicit the Use of In-Context World Representations in LLMs21 Sep 2026
- 2026Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching10 Aug 2026
- 2026TraceViT: Grounded Trace Supervision for Visual Abstract Reasoning31 Jul 2026
- 2026Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers16 Jun 2026
- 2026LT2: Linear-Time Looped Transformers20 May 2026
- 2026Investigating Single-Block Recurrence in Vision Transformers for Image Recognition11 May 2026
- 2026ELT: Elastic Looped Transformers for Visual Generation10 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2024Tackling the Abstraction and Reasoning Corpus with Vision Transformers: the Importance of 2D Representation, Positions, and Objects
- 2026DeepLatent: Think with Images via Parallel Latent Visual Reasoning
- 2026Counting Circuits: Mechanistic Interpretability of Visual Reasoning in Large Vision-Language Models
- 2026LoopMTP: A looped transformer guided by latent multi-token prediction
- 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
- 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens
- 2025ViT-Linearizer: Distilling Quadratic Knowledge into Linear-Time Vision Models
- 2023Training-Free Acceleration of ViTs with Delayed Spatial Merging