mlsanityData to 7 Oct 2026

PUZZLES: A Benchmark for Neural Algorithmic Reasoning

arXiv 2407.00401 ↗First posted 29 Jun 2024cs.LG cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

Algorithmic reasoning is a fundamental cognitive ability that plays a pivotal role in problem-solving and decision-making processes. Reinforcement Learning (RL) has demonstrated remarkable proficiency in tasks such as motor control, handling perceptual input, and managing stochastic environments. These advancements have been enabled in part by the availability of benchmarks. In this work we introduce PUZZLES, a benchmark based on Simon Tatham's Portable Puzzle Collection, aimed at fostering progress in algorithmic and logical reasoning in RL. PUZZLES contains 40 diverse logic puzzles of adjustable sizes and varying levels of complexity; many puzzles also feature a diverse set of additional configuration parameters. The 40 puzzles provide detailed information on the strengths and generalization capabilities of RL agents. Furthermore, we evaluate various RL algorithms on PUZZLES, providing baseline comparisons and demonstrating the potential for future research. All the software, including the environment, is available at https://github.com/ETH-DISCO/rlp.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 41 of its 75 references that are in the map. First posted on arXiv 29 Jun 2024.

Builds on

41 of 75 references.

References · 41

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  5. 2015Trust Region Policy Optimizationcited by 2,503
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  8. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  9. 2016OpenAI Gymcited by 1,684
  10. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  11. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  12. 2023Mastering Diverse Domains through World Modelscited by 1,193
29 more
  1. 2018DeepMind Control Suitecited by 935
  2. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  3. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  4. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  5. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  6. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
  7. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  8. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
  9. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  10. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 307
  11. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  12. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  13. 2019OpenSpiel: A Framework for Reinforcement Learning in Gamescited by 180
  14. 2016Deep Reinforcement Learning from Self-Play in Imperfect-Information Gamescited by 153
  15. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 129
  16. 2022The CLRS Algorithmic Reasoning Benchmarkcited by 103
  17. 2021Neural Algorithmic Reasoningcited by 92
  18. 2020Suphx: Mastering Mahjong with Deep Reinforcement Learningcited by 55
  19. 2021Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021)cited by 55
  20. 2019Causal Reasoning from Meta-reinforcement Learningcited by 53
  21. 2019Neural Logic Reinforcement Learningcited by 53
  22. 2022Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Roboticscited by 23
  23. 2022MERIt: Meta-Path Guided Contrastive Learning for Logical Reasoningcited by 22
  24. 2021Neural Algorithmic Reasoners are Implicit Plannerscited by 21
  25. 2022Intelligent problem-solving as integrated hierarchical reinforcement learningcited by 21
  26. 2020Strong Generalization and Efficiency in Neural Programscited by 12
  27. 2015Giraffe: Using Deep Reinforcement Learning to Play Chesscited by 11
  28. 2022Continuous Neural Algorithmic Plannerscited by 4
  29. 2022Rethinking Reinforcement Learning based Logic Synthesiscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 3.

12 papers.

  1. 2025iVISPAR -- An Interactive Visual-Spatial Reasoning Benchmark for VLMscited by 24
  2. 2025Primal-Dual Neural Algorithmic Reasoningcited by 8
  3. 2025PuzzleWorld: A Benchmark for Multimodal, Open-Ended Reasoning in Puzzlehuntscited by 7
  4. 2025Benchmarking World-Model Learning with Environment-Level Queriescited by 7
  5. 2025The Jumping Reasoning Curve? Tracking the Evolution of Reasoning Performance in GPT-[n] and o-[n] Models on Multimodal Puzzlescited by 4
  6. 2024Sliding Puzzles Gym: A Scalable Benchmark for State Representation in Visual Reinforcement Learningcited by 3
  7. 2025Reasoning Effort and Problem Complexity: A Scaling Analysis in LLMscited by 3
  8. 2026TopoBench: Benchmarking LLMs on Hard Topological Reasoningcited by 3
  9. 2025Beyond Interpolation: Extrapolative Reasoning with Reinforcement Learning and Graph Neural Networkscited by 1
  10. 2026Measuring AI Reasoning: A Guide for Researcherscited by 1
  11. 2025GenEscape: Hierarchical Multi-Agent Generation of Escape Room Puzzlescited by 0
  12. 2026How Well Do LLMs Perform on the Simplest Long-Chain Reasoning Tasks: An Empirical Study on the Equivalence Class Problemcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey of Reinforcement Learning for Large Reasoning Models
  2. 2025LogicPuzzleRL: Cultivating Robust Mathematical Reasoning in LLMs via Reinforcement Learning
  3. 2025PuzzleJAX: A Benchmark for Reasoning and Learning
  4. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning
  5. 2024Puzzle Solving using Reasoning of Large Language Models: A Survey
  6. 2026Witness: Discovery, Deciphering, and Epiphany in Interactive Puzzle Environments
  7. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewards
  8. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failures