# Retrieval-Guided Reinforcement Learning for Boolean Circuit Minimization

arXiv: https://arxiv.org/abs/2401.12205 · HTML: https://mlsanity.com/paper/2401.12205/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 19 of its 44 references that are in the map.
- First posted on arXiv 22 Jan 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [VeriGen: A Large Language Model for Verilog Code Generation](https://mlsanity.com/paper/2308.00708/) (2023)
- [What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Study](https://mlsanity.com/paper/2006.05990/) (2020)
- [Retrieval-Augmented Reinforcement Learning](https://mlsanity.com/paper/2202.08417/) (2022)
- [DRiLLS: Deep Reinforcement Learning for Logic Synthesis](https://mlsanity.com/paper/1911.04021/) (2019)
- [MaskPlace: Fast Chip Placement via Reinforced Visual Representation Learning](https://mlsanity.com/paper/2211.13382/) (2022)
- [ChiPFormer: Transferable Chip Placement via Offline Decision Transformer](https://mlsanity.com/paper/2306.14744/) (2023)
- [DeepGate: Learning Neural Representations of Logic Gates](https://mlsanity.com/paper/2111.14616/) (2021)
- [DeepGate2: Functionality-Aware Circuit Representation Learning](https://mlsanity.com/paper/2305.16373/) (2023)
- [OpenABC-D: A Large-Scale Dataset For Machine Learning Guided Integrated Circuit Synthesis](https://mlsanity.com/paper/2110.11292/) (2021)
- [Developing Synthesis Flows Without Human Knowledge](https://mlsanity.com/paper/1804.05714/) (2018)
- [Large-Scale Retrieval for Reinforcement Learning](https://mlsanity.com/paper/2206.05314/) (2022)
- [GLSearch: Maximum Common Subgraph Detection via Learning to Search](https://mlsanity.com/paper/2002.03129/) (2020)
- [Neural Circuit Synthesis from Specification Patterns](https://mlsanity.com/paper/2107.11864/) (2021)
- [FlowTune: End-to-end Automatic Logic Optimization Exploration via Domain-specific Multi-armed Bandit](https://mlsanity.com/paper/2202.07721/) (2022)

## Built on by (most cited first, 10 of 10)

- [ORFS-agent: Tool-Using Agents for Chip Design Optimization](https://mlsanity.com/paper/2506.08332/) (2025)
- [Retrieval-Enhanced Machine Learning: Synthesis and Opportunities](https://mlsanity.com/paper/2407.12982/) (2024)
- [NetDeTox: Adversarial and Efficient Evasion of Hardware-Security GNNs via RL-LLM Orchestration](https://mlsanity.com/paper/2512.00119/) (2025)
- [Dr. RTL: Autonomous Agentic RTL Optimization through Tool-Grounded Self-Improvement](https://mlsanity.com/paper/2604.14989/) (2026)
- [That Chip Has Sailed: A Critique of Unfounded Skepticism Around AI for Chip Design](https://mlsanity.com/paper/2411.10053/) (2024)
- [VeriLoC: Line-of-Code Level Prediction of Hardware Design Quality from Verilog Code](https://mlsanity.com/paper/2506.07239/) (2025)
- [Logic Synthesis Optimization with Predictive Self-Supervision via Causal Transformers](https://mlsanity.com/paper/2409.10653/) (2024)
- [LTLf Learning Meets Boolean Set Cover](https://mlsanity.com/paper/2509.24616/) (2025)
- [ASCENT: Amplifying Power Side-Channel Resilience via Learning & Monte-Carlo Tree Search](https://mlsanity.com/paper/2406.19549/) (2024)
- [Natural Synthesis: Outperforming Reactive Synthesis Tools with Large Reasoning Models](https://mlsanity.com/paper/2605.15131/) (2026)

## Abstract (arXiv)

Logic synthesis, a pivotal stage in chip design, entails optimizing chip specifications encoded in hardware description languages like Verilog into highly efficient implementations using Boolean logic gates. The process involves a sequential application of logic minimization heuristics (``synthesis recipe"), with their arrangement significantly impacting crucial metrics such as area and delay. Addressing the challenge posed by the broad spectrum of design complexities - from variations of past designs (e.g., adders and multipliers) to entirely novel configurations (e.g., innovative processor instructions) - requires a nuanced `synthesis recipe` guided by human expertise and intuition. This study conducts a thorough examination of learning and search techniques for logic synthesis, unearthing a surprising revelation: pre-trained agents, when confronted with entirely novel designs, may veer off course, detrimentally affecting the search trajectory. We present ABC-RL, a meticulously tuned α parameter that adeptly adjusts recommendations from pre-trained agents during the search process. Computed based on similarity scores through nearest neighbor retrieval from the training dataset, ABC-RL yields superior synthesis recipes tailored for a wide array of hardware designs. Our findings showcase substantial enhancements in the Quality-of-result (QoR) of synthesized circuits, boasting improvements of up to 24.8% compared to state-of-the-art techniques. Furthermore, ABC-RL achieves an impressive up to 9x reduction in runtime (iso-QoR) when compared to current state-of-the-art methodologies.