mlsanityData to 14 Oct 2026

Large Language Model (LLM) as a System of Multiple Expert Agents: An Approach to solve the Abstraction and Reasoning Corpus (ARC) Challenge

arXiv 2310.05146 ↗First posted 8 Oct 2023cs.AI cs.CLcited by 11

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

We attempt to solve the Abstraction and Reasoning Corpus (ARC) Challenge using Large Language Models (LLMs) as a system of multiple expert agents. Using the flexibility of LLMs to be prompted to do various novel tasks using zero-shot, few-shot, context-grounded prompting, we explore the feasibility of using LLMs to solve the ARC Challenge. We firstly convert the input image into multiple suitable text-based abstraction spaces. We then utilise the associative power of LLMs to derive the input-output relationship and map this to actions in the form of a working program, similar to Voyager / Ghost in the MineCraft. In addition, we use iterative environmental feedback in order to guide LLMs to solve the task. Our proposed approach achieves 50 solves out of 111 training set problems (45%) with just three abstraction spaces - grid, object and pixel - and we believe that with more abstraction spaces and learnable actions, we will be able to solve more.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 19 references that are in the map. First posted on arXiv 8 Oct 2023.

Builds on

17 of 19 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023An Approach to Solving the Abstraction and Reasoning Corpus (ARC) Challenge6 Jun
  2. 2023LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations26 May
  3. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models8 Mar
  4. 2023Abstract Visual Reasoning Enabled by Language7 Mar

Other references · 13

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  5. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  6. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  7. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  8. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 289
  9. 2023Large Language Models as General Pattern Machinescited by 197
  10. 2021Communicating Natural Programs to Humans and Machinescited by 35
  11. 2022Graphs, Constraints, and Search for the Abstraction and Reasoning Corpuscited by 32
  12. 2021First Steps of an Approach to the ARC Challenge based on Descriptive Grid Models and the Minimum Description Length Principlecited by 13
1 more
  1. 2023Unraveling the ARC Puzzle: Mimicking Human Solutions with Object-Centric Decision Transformercited by 12

Built on by

Cited by year

Citing papers per year: 2024: 6; 2025: 5.

11 papers.

  1. 2024Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examplescited by 30
  2. 2024Polymath: A Challenging Multi-modal Mathematical Reasoning Benchmarkcited by 22
  3. 2024IW-Bench: Evaluating Large Multimodal Models for Converting Image-to-Webcited by 16
  4. 2024HYSYNTH: Context-Free LLM Approximation for Guiding Program Synthesiscited by 15
  5. 2025The Stochastic Parrot on LLM's Shoulder: A Summative Assessment of Physical Concept Understandingcited by 11
  6. 2025ARC-GEN: A Mimetic Procedural Benchmark Generator for the Abstraction and Reasoning Corpuscited by 10
  7. 2025From Standalone LLMs to Integrated Intelligence: A Survey of Compound Al Systemscited by 6
  8. 2025Understanding LLMs' Fluid Intelligence Deficiency: An Analysis of the ARC Taskcited by 4
  9. 2024AbstractBeam: Enhancing Bottom-Up Program Synthesis using Library Learningcited by 2
  10. 2024Enhancing Analogical Reasoning in the Abstraction and Reasoning Corpus via Model-Based RLcited by 2
  11. 2025Solomonoff-Inspired Hypothesis Ranking with LLMs for Prediction Under Uncertaintycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Product of Experts with LLMs: Boosting Performance on ARC Is a Matter of Perspective
  2. 2025An Analysis of Architectural Impact on LLM-based Abstract Visual Reasoning: A Systematic Benchmark on RAVEN-FAIR
  3. 2023Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaboration
  4. 2024Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models
  5. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users
  6. 2025Be My Eyes: Extending Large Language Models to New Modalities Through Multi-Agent Collaboration
  7. 2025LLM+AL: Bridging Large Language Models and Action Languages for Complex Reasoning about Actions
  8. 2024Intelligence Analysis of Language Models