mlsanityData to 7 Oct 2026

Fast and flexible: Human program induction in abstract reasoning tasks

arXiv 2103.05823 ↗First posted 10 Mar 2021cs.HC cs.AI cs.LGcited by 33

Not on the map. Why?

Abstract (arXiv)

The Abstraction and Reasoning Corpus (ARC) is a challenging program induction dataset that was recently proposed by Chollet (2019). Here, we report the first set of results collected from a behavioral study of humans solving a subset of tasks from ARC (40 out of 1000). Although this subset of tasks contains considerable variation, our results showed that humans were able to infer the underlying program and generate the correct test output for a novel test input example, with an average of 80% of tasks solved per participant, and with 65% of tasks being solved by more than 80% of participants. Additionally, we find interesting patterns of behavioral consistency and variability within the action sequences during the generation process, the natural language descriptions to describe the transformations for each task, and the errors people made. Our findings suggest that people can quickly and reliably determine the relevant features and properties of a task to compose a correct solution. Future modeling work could incorporate these findings, potentially by connecting the natural language descriptions we collected here to the underlying semantics of ARC.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 4 of its 25 references that are in the map. First posted on arXiv 10 Mar 2021.

Builds on

4 of 25 references.

References · 4

  1. 2020DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learningcited by 159
  2. 2021Abstraction and Analogy-Making in Artificial Intelligencecited by 103
  3. 2017Learning with Latent Languagecited by 75
  4. 2019Human few-shot learning of compositional instructionscited by 47

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 11; 2024: 11; 2025: 5; 2026: 3.

33 papers.

Sort:

By area:

  1. 2025ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systemscited by 171
  2. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 119
  3. 2023The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domaincited by 97
  4. 2023Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinementcited by 89
  5. 2023Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Taskscited by 54
  6. 2024Code Repair with LLMs gives an Exploration-Exploitation Tradeoffcited by 50
  7. 2021Communicating Natural Programs to Humans and Machinescited by 35
  8. 2022Graphs, Constraints, and Search for the Abstraction and Reasoning Corpuscited by 32
  9. 2024Reasoning Abilities of Large Language Models: In-Depth Analysis on the Abstraction and Reasoning Corpuscited by 32
  10. 2024H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmarkcited by 32
  11. 2024CodeIt: Self-Improving Language Models with Prioritized Hindsight Replaycited by 23
  12. 2023Abstract Visual Reasoning Enabled by Languagecited by 15
21 more
  1. 2026LoopViT: Scaling Visual ARC with Looped Transformerscited by 14
  2. 2023ANPL: Towards Natural Programming with Interactive Decompositioncited by 13
  3. 2023Unraveling the ARC Puzzle: Mimicking Human Solutions with Object-Centric Decision Transformercited by 12
  4. 2023A Neurodiversity-Inspired Solver for the Abstraction & Reasoning Corpus (ARC) Using Visual Imagery and Program Synthesiscited by 9
  5. 2023Solving ARC visual analogies with neural embeddings and vector arithmetic: A generalized methodcited by 8
  6. 2023A Divide-Align-Conquer Strategy for Program Synthesiscited by 7
  7. 2024Do Large Language Models Solve ARC Visual Analogies Like People Do?cited by 7
  8. 2024ARCLE: The Abstraction and Reasoning Corpus Learning Environment for Reinforcement Learningcited by 7
  9. 2022Disentangling Abstraction from Statistical Pattern Matching in Human and Machine Learningcited by 6
  10. 2023Tackling the Abstraction and Reasoning Corpus (ARC) with Object-centric Models and the MDL Principlecited by 6
  11. 2024Program Synthesis using Inductive Logic Programming for the Abstraction and Reasoning Corpuscited by 4
  12. 2024Connecting the Dots: Evaluating Abstract Reasoning Capabilities of LLMs Using the New York Times Connections Word Gamecited by 3
  13. 2024Enhancing Analogical Reasoning in the Abstraction and Reasoning Corpus via Model-Based RLcited by 2
  14. 2025NSA: Neuro-symbolic ARC Challengecited by 2
  15. 2025From Reasoning to Generalization: Knowledge-Augmented LLMs for ARC Benchmarkcited by 2
  16. 2024Diffusion-Based Offline RL for Improved Decision-Making in Augmented ARC Taskcited by 1
  17. 2025Vector Symbolic Algebras for the Abstraction and Reasoning Corpuscited by 1
  18. 2026Online library learning in human visual puzzle solvingcited by 1
  19. 2024Addressing and Visualizing Misalignments in Human Task-Solving Trajectoriescited by 0
  20. 2025Analysis of Error Sources in LLM-based Hypothesis Search for Few-Shot Rule Inductioncited by 0
  21. 2026Prospective Compression in Human Abstraction Learningcited by 0