mlsanityData to 7 Oct 2026

What makes math problems hard for reinforcement learning: a case study

arXiv 2408.15332 ↗First posted 27 Aug 2024cs.LG cs.AI math.COcited by 9

Not on the map. Why?

Abstract (arXiv)

Using a long-standing conjecture from combinatorial group theory, we explore, from multiple perspectives, the challenges of finding rare instances carrying disproportionately high rewards. Based on lessons learned in the context defined by the Andrews-Curtis conjecture, we propose algorithmic enhancements and a topological hardness measure with implications for a broad class of search problems. As part of our study, we also address several open mathematical questions. Notably, we demonstrate the length reducibility of all but two presentations in the Akbulut-Kirby series (1981), and resolve various potential counterexamples in the Miller-Schupp series (1991), including three infinite subfamilies.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 22 of its 59 references that are in the map. First posted on arXiv 27 Aug 2024.

Builds on

22 of 59 references.

References · 22

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  5. 2023Large Language Modelscited by 774
  6. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  7. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  8. 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 414
  9. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  10. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
  11. 2016Using the Output Embedding to Improve Language Modelscited by 266
  12. 2020Never Give Up: Learning Directed Exploration Strategiescited by 210
10 more
  1. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 207
  2. 2020Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPOcited by 184
  3. 2023ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematicscited by 180
  4. 2016Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modelingcited by 87
  5. 2024Lean-STaR: Learning to Interleave Thinking and Provingcited by 64
  6. 2020giotto-tda: A Topological Data Analysis Toolkit for Machine Learning and Data Explorationcited by 55
  7. 2022A2C is a special case of PPOcited by 16
  8. 2022Peano: Learning Formal Mathematical Reasoningcited by 15
  9. 2020The Simplex Tree: an Efficient Data Structure for General Simplicial Complexescited by 14
  10. 2016Distance Metric Ensemble Learning and the Andrews-Curtis Conjecturecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 5.

9 papers.

  1. 2025CayleyPy RL: Pathfinding and Reinforcement Learning on Cayley Graphscited by 5
  2. 2026Flow-based Extremal Mathematical Structure Discoverycited by 3
  3. 2025Diffusion Models for Cayley Graphscited by 1
  4. 2025CayleyPy Growth: Efficient growth computations and hundreds of new conjectures on Cayley graphs (Brief version)cited by 1
  5. 2025An AI enhanced approach to the tree unimodality conjecturecited by 1
  6. 2026Discovering mathematical concepts through a multi-agent systemcited by 1
  7. 2026TriSearch: Learning to Optimize Triangulations via Bistellar Flipscited by 0
  8. 2026The Two-Hump Problem: Bridging the Difficulty Gap in Mathematical Reinforcement Learningcited by 0
  9. 2026SAGE: Mitigating Long-Horizon Reasoning Biases via Topological Guidancecited by 0