mlsanityData to 14 Oct 2026

Hierarchical Reinforcement Learning for Sparse-Reward Search in Commutative Algebra

arXiv 2606.22922 ↗First posted 22 Jun 2026cs.LG cs.AI math.ACcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Applying machine learning techniques to solving long-standing mathematical conjectures can be particularly challenging due to their extreme reward sparsity. As an illustrative example, we consider Kalai's algebraic Hirsch conjecture and recast the construction of its counterexamples as a sparse-reward reinforcement learning problem on graphs. We propose a constrained options-based HRL framework with an equivariant graph neural network policy, which allows us to learn useful temporal abstractions for this task. We evaluate our approach over a wide range of degrees and demonstrate that it consistently outperforms classical RL algorithms as well as greedy search. By exploiting the hierarchical structure of the problem, we effectively provide a first-of-its-kind application of HRL to a problem in commutative algebra.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 29 references that are in the map. First posted on arXiv 22 Jun 2026.

Builds on

14 of 29 references.

References · 14

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Graph Attention Networkscited by 8,923
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  5. 2017Hindsight Experience Replaycited by 1,136
  6. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 713
  7. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  8. 2016The Option-Critic Architecturecited by 448
  9. 2020First return, then explorecited by 266
  10. 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 171
  11. 2017Eigenoption Discovery through the Deep Successor Representationcited by 86
  12. 2021Heuristic-Guided Reinforcement Learningcited by 42
2 more
  1. 2017Learnings Options End-to-End for Continuous Action Taskscited by 24
  2. 2025Advancing Geometry with AI: Multi-agent Generation of Polytopescited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026RL4RLA: Teaching ML to Discover Randomized Linear Algebra Algorithms Through Curriculum Design and Graph-Based Search
  2. 2026Reinforcement Learning for Symbolic Equation Solving
  3. 2025Reinforcement learning for graph theory, Parallelizing Wagner's approach
  4. 2026A Representation Optimization Dichotomy, Lie-Algebraic Policy Optimization
  5. 2026RLGT: A reinforcement learning framework for extremal graph theory
  6. 2025Discovering Hidden Algebraic Structures via Transformers with Rank-Aware Beam GRPO
  7. 2025MatRL: Provably Generalizable Iterative Algorithm Discovery via Monte-Carlo Tree Search
  8. 2025ReFill: Reinforcement Learning for Fill-In Minimization