mlsanityData to 7 Oct 2026

LoopViT: Scaling Visual ARC with Looped Transformers

arXiv 2602.02156 ↗First posted 2 Feb 2026cs.CVcited by 14

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Recent advances in visual reasoning have leveraged vision transformers to tackle the ARC-AGI benchmark. However, we argue that the feed-forward architecture, where computational depth is strictly bound to parameter size, falls short of capturing the iterative, algorithmic nature of human induction. In this work, we propose a recursive architecture called Loop-ViT, which decouples reasoning depth from model capacity through weight-tied recurrence. Loop-ViT iterates a weight-tied Hybrid Block, combining local convolutions and global attention, to form a latent chain of thought. Crucially, we introduce a parameter-free Dynamic Exit mechanism based on predictive entropy: the model halts inference when its internal state ``crystallizes" into a low-uncertainty attractor. Empirical results on the ARC-AGI-1 benchmark validate this perspective: our 18M model achieves 65.8% accuracy, outperforming massive 73M-parameter ensembles. These findings demonstrate that adaptive iterative computation offers a far more efficient scaling axis for visual reasoning than simply increasing network width. The code is available at https://github.com/WenjieShu/LoopViT.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 41 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

28 of 41 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024ARC Prize 2024: Technical Report5 Dec
  2. 2024Combining Induction and Transduction for Abstract Reasoning4 Nov

Other references · 26

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  4. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  6. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  7. 2019Root Mean Square Layer Normalizationcited by 1,425
  8. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  9. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
  10. 2024Mixture-of-Depths: Dynamically allocating compute in transformer-based language modelscited by 242
  11. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 223
  12. 2025Hierarchical Reasoning Modelcited by 167
14 more
  1. 2025Scaling Latent Reasoning via Looped Language Modelscited by 145
  2. 2023Looped Transformers are Better at Learning Learning Algorithmscited by 135
  3. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 119
  4. 2021Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networkscited by 108
  5. 2023The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domaincited by 97
  6. 2024Looped Transformers for Length Generalizationcited by 91
  7. 2024Code Repair with LLMs gives an Exploration-Exploitation Tradeoffcited by 50
  8. 2025PonderLM: Pretraining Language Models to Ponder in Continuous Spacecited by 44
  9. 2021Fast and flexible: Human program induction in abstract reasoning taskscited by 33
  10. 2024H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmarkcited by 32
  11. 2024Addressing the Abstraction and Reasoning Corpus via Procedural Example Generationcited by 31
  12. 2023LGViT: Dynamic Early Exiting for Accelerating Vision Transformercited by 30
  13. 2025ARC Is a Vision Problem!cited by 23
  14. 2025Understanding and Benchmarking Artificial Intelligence: OpenAI's o3 Is Not AGIcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2024Tackling the Abstraction and Reasoning Corpus with Vision Transformers: the Importance of 2D Representation, Positions, and Objects
  2. 2026DeepLatent: Think with Images via Parallel Latent Visual Reasoning
  3. 2026Counting Circuits: Mechanistic Interpretability of Visual Reasoning in Large Vision-Language Models
  4. 2026LoopMTP: A looped transformer guided by latent multi-token prediction
  5. 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
  6. 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens
  7. 2025ViT-Linearizer: Distilling Quadratic Knowledge into Linear-Time Vision Models
  8. 2023Training-Free Acceleration of ViTs with Delayed Spatial Merging