mlsanityData to 14 Oct 2026

CircuitBuilder: From Polynomials to Circuits via Reinforcement Learning

arXiv 2603.17075 ↗First posted 17 Mar 2026cs.LG cs.AI cs.CCcited by 1

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Motivated by auto-proof generation and Valiant's VP vs. VNP conjecture, we study the problem of discovering efficient arithmetic circuits to compute polynomials, using addition and multiplication gates. We formulate this problem as a single-player game, where an RL agent attempts to build the circuit within a fixed number of operations. We implement an AlphaZero-style training loop and compare two approaches: Proximal Policy Optimization with Monte Carlo Tree Search (PPO+MCTS) and Soft Actor-Critic (SAC). SAC achieves the highest success rates on two-variable targets, while PPO+MCTS scales to three variables and demonstrates steady improvement on harder instances. These results suggest that polynomial circuit synthesis is a compact, verifiable setting for studying self-improving search policies.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 3 of its 11 references that are in the map. First posted on arXiv 17 Mar 2026.

Builds on

3 of 11 references.

References · 3

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2019Soft Actor-Critic for Discrete Action Settingscited by 174

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026FactorLibrary: From Polynomials to Circuits via Recursive Subgoalscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning Fast Monomial Orders for Gr"obner Basis Computations
  2. 2024Scalable and Effective Arithmetic Tree Generation for Adder and Multiplier Designs
  3. 2024Retrieval-Guided Reinforcement Learning for Boolean Circuit Minimization
  4. 2026EvolveGen: Algorithmic Level Hardware Model Checking Benchmark Generation through Reinforcement Learning
  5. 2026MechRL: Reinforcement Learning Agents Perform Circuit Discovery for Mechanistic Interpretability
  6. 2026Advancing Mathematics Research with AI-Driven Formal Proof Search
  7. 2024GraCo -- A Graph Composer for Integrated Circuits
  8. 2026AstRL: Analog and Mixed-Signal Circuit Synthesis with Deep Reinforcement Learning