mlsanityData to 14 Oct 2026

ARCLE: The Abstraction and Reasoning Corpus Learning Environment for Reinforcement Learning

arXiv 2407.20806 ↗First posted 30 Jul 2024cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

This paper introduces ARCLE, an environment designed to facilitate reinforcement learning research on the Abstraction and Reasoning Corpus (ARC). Addressing this inductive reasoning benchmark with reinforcement learning presents these challenges: a vast action space, a hard-to-reach goal, and a variety of tasks. We demonstrate that an agent with proximal policy optimization can learn individual tasks through ARCLE. The adoption of non-factorial policies and auxiliary losses led to performance enhancements, effectively mitigating issues associated with action spaces and goal attainment. Based on these insights, we propose several research directions and motivations for using ARCLE, including MAML, GFlowNets, and World Models.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 51 of its 67 references that are in the map. First posted on arXiv 30 Jul 2024.

Builds on

51 of 67 references.

References · 51

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  6. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  7. 2020Learning to summarize from human feedbackcited by 2,588
  8. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  9. 2023Mastering Diverse Domains through World Modelscited by 1,253
  10. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  11. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  12. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
38 more
  1. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  2. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 392
  3. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 303
  4. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 274
  5. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 251
  6. 2019RAVEN: A Dataset for Relational and Analogical Visual rEasoNingcited by 226
  7. 2019R2D2: Repeatable and Reliable Detector and Descriptorcited by 207
  8. 2020The NetHack Learning Environmentcited by 198
  9. 2023Large Language Models as General Pattern Machinescited by 197
  10. 2022Biological Sequence Design with GFlowNetscited by 174
  11. 2018Measuring abstract reasoning in neural networkscited by 166
  12. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
  13. 2022Learning GFlowNets from partial episodes for improved convergence and stabilitycited by 132
  14. 2023A Tutorial on Meta-Reinforcement Learningcited by 108
  15. 2022Multi-Objective GFlowNetscited by 101
  16. 2023The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domaincited by 101
  17. 2016Playing FPS Games with Deep Reinforcement Learningcited by 95
  18. 2023LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representationscited by 82
  19. 2023Robust Scheduling with GFlowNetscited by 69
  20. 2021DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularizationcited by 66
  21. 2023Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Taskscited by 55
  22. 2019Learning to Make Analogies by Contrasting Abstract Relational Structurecited by 53
  23. 2023Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNetscited by 40
  24. 2020Learning Representations that Support Extrapolationcited by 36
  25. 2021Communicating Natural Programs to Humans and Machinescited by 35
  26. 2021Fast and flexible: Human program induction in abstract reasoning taskscited by 33
  27. 2022A Review of Emerging Research Directions in Abstract Visual Reasoningcited by 33
  28. 2022Graphs, Constraints, and Search for the Abstraction and Reasoning Corpuscited by 32
  29. 2024Reasoning Abilities of Large Language Models: In-Depth Analysis on the Abstraction and Reasoning Corpuscited by 32
  30. 2021Neural-guided, Bidirectional Program Search for Abstraction and Reasoningcited by 23
  31. 2024CodeIt: Self-Improving Language Models with Prioritized Hindsight Replaycited by 23
  32. 2020Machine Number Sense: A Dataset of Visual Arithmetic Problems for Abstract and Relational Reasoningcited by 18
  33. 2023Abstract Visual Reasoning Enabled by Languagecited by 15
  34. 2023Unraveling the ARC Puzzle: Mimicking Human Solutions with Object-Centric Decision Transformercited by 12
  35. 2023A Neurodiversity-Inspired Solver for the Abstraction & Reasoning Corpus (ARC) Using Visual Imagery and Program Synthesiscited by 9
  36. 2023Solving ARC visual analogies with neural embeddings and vector arithmetic: A generalized methodcited by 8
  37. 2023A Divide-Align-Conquer Strategy for Program Synthesiscited by 7
  38. 2021PQA: Perceptual Question Answeringcited by 6

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 1; 2026: 2.

7 papers.

  1. 2026ResearchGym: Evaluating Language Model Agents on Real-World AI Researchcited by 22
  2. 2024Tackling the Abstraction and Reasoning Corpus with Vision Transformers: the Importance of 2D Representation, Positions, and Objectscited by 10
  3. 2025Think Visually, Reason Textually: Vision-Language Synergy in ARCcited by 6
  4. 2024Enhancing Analogical Reasoning in the Abstraction and Reasoning Corpus via Model-Based RLcited by 2
  5. 2024Diffusion-Based Offline RL for Improved Decision-Making in Augmented ARC Taskcited by 1
  6. 2024Addressing and Visualizing Misalignments in Human Task-Solving Trajectoriescited by 0
  7. 2026JaxARC: A High-Performance JAX-based Environment for Abstraction and Reasoning Researchcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use
  2. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Survey
  3. 2026Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning
  4. 2025Agent-as-Tool: A Study on the Hierarchical Decision Making with Reinforcement Learning
  5. 2025Context-Sensitive Abstractions for Reinforcement Learning with Parameterized Actions
  6. 2025AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning
  7. 2025Reasoning Core: A Scalable RL Environment for LLM Symbolic Reasoning
  8. 2025PARL: Prompt-based Agents for Reinforcement Learning