# Abstract Visual Reasoning Enabled by Language

arXiv: https://arxiv.org/abs/2303.04091 · HTML: https://mlsanity.com/paper/2303.04091/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 7 of its 13 references that are in the map.
- First posted on arXiv 7 Mar 2023.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 7 of 7)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Communicating Natural Programs to Humans and Machines](https://mlsanity.com/paper/2106.07824/) (2021)
- [Fast and flexible: Human program induction in abstract reasoning tasks](https://mlsanity.com/paper/2103.05823/) (2021)
- [Graphs, Constraints, and Search for the Abstraction and Reasoning Corpus](https://mlsanity.com/paper/2210.09880/) (2022)
- [Neural-guided, Bidirectional Program Search for Abstraction and Reasoning](https://mlsanity.com/paper/2110.11536/) (2021)
- [ViRel: Unsupervised Visual Relations Discovery with Graph-level Analogy](https://mlsanity.com/paper/2207.00590/) (2022)

## Built on by (most cited first, 15 of 15)

- [LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations](https://mlsanity.com/paper/2305.18354/) (2023)
- [What is the Visual Cognition Gap between Humans and Multimodal LLMs?](https://mlsanity.com/paper/2406.10424/) (2024)
- [Limits of Transformer Language Models on Learning to Compose Algorithms](https://mlsanity.com/paper/2402.05785/) (2024)
- [Learning Differentiable Logic Programs for Abstract Visual Reasoning](https://mlsanity.com/paper/2307.00928/) (2023)
- [Large Language Model (LLM) as a System of Multiple Expert Agents: An Approach to solve the Abstraction and Reasoning Corpus (ARC) Challenge](https://mlsanity.com/paper/2310.05146/) (2023)
- [Tackling the Abstraction and Reasoning Corpus with Vision Transformers: the Importance of 2D Representation, Positions, and Objects](https://mlsanity.com/paper/2410.06405/) (2024)
- [Towards Learning Abductive Reasoning using VSA Distributed Representations](https://mlsanity.com/paper/2406.19121/) (2024)
- [ARCLE: The Abstraction and Reasoning Corpus Learning Environment for Reinforcement Learning](https://mlsanity.com/paper/2407.20806/) (2024)
- [Towards Learning to Reason: Comparing LLMs with Neuro-Symbolic on Arithmetic Relations in Abstract Reasoning](https://mlsanity.com/paper/2412.05586/) (2024)
- [Don't throw the baby out with the bathwater: How and why deep learning for ARC](https://mlsanity.com/paper/2506.14276/) (2025)
- [Can Large Reasoning Models do Analogical Reasoning under Perceptual Uncertainty?](https://mlsanity.com/paper/2503.11207/) (2025)
- [From Reasoning to Generalization: Knowledge-Augmented LLMs for ARC Benchmark](https://mlsanity.com/paper/2505.17482/) (2025)
- [Replicating Human Social Perception in Generative AI: Evaluating the Valence-Dominance Model](https://mlsanity.com/paper/2503.04842/) (2025)
- [JaxARC: A High-Performance JAX-based Environment for Abstraction and Reasoning Research](https://mlsanity.com/paper/2601.17564/) (2026)
- [Predictive Reasoning with Augmented Anomaly Contrastive Learning for Compositional Visual Relations](https://mlsanity.com/paper/2603.01125/) (2026)

## Abstract (arXiv)

While artificial intelligence (AI) models have achieved human or even superhuman performance in many well-defined applications, they still struggle to show signs of broad and flexible intelligence. The Abstraction and Reasoning Corpus (ARC), a visual intelligence benchmark introduced by Franccois Chollet, aims to assess how close AI systems are to human-like cognitive abilities. Most current approaches rely on carefully handcrafted domain-specific program searches to brute-force solutions for the tasks present in ARC. In this work, we propose a general learning-based framework for solving ARC. It is centered on transforming tasks from the vision to the language domain. This composition of language and vision allows for pre-trained models to be leveraged at each stage, enabling a shift from handcrafted priors towards the learned priors of the models. While not yet beating state-of-the-art models on ARC, we demonstrate the potential of our approach, for instance, by solving some ARC tasks that have not been solved previously.