# PrefixRL: Optimization of Parallel Prefix Circuits using Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2205.07000 · HTML: https://mlsanity.com/paper/2205.07000/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 27 references that are in the map.
- First posted on arXiv 14 May 2022.

## Builds on (most cited first, 5 of 5)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Multi-Objective Deep Reinforcement Learning](https://mlsanity.com/paper/1610.02707/) (2016)
- [Chip Placement with Deep Reinforcement Learning](https://mlsanity.com/paper/2004.10746/) (2020)
- [Cross-layer Optimization for High Speed Adders: A Pareto Driven Machine Learning Approach](https://mlsanity.com/paper/1807.07023/) (2018)

## Built on by (most cited first, 18 of 18)

- [ChipNeMo: Domain-Adapted LLMs for Chip Design](https://mlsanity.com/paper/2311.00176/) (2023)
- [Scientists' Perspectives on the Potential for Generative AI in their Fields](https://mlsanity.com/paper/2304.01420/) (2023)
- [General Intelligence Requires Rethinking Exploration](https://mlsanity.com/paper/2211.07819/) (2022)
- [Entity-based Reinforcement Learning for Autonomous Cyber Defence](https://mlsanity.com/paper/2410.17647/) (2024)
- [Scalable and Effective Arithmetic Tree Generation for Adder and Multiplier Designs](https://mlsanity.com/paper/2405.06758/) (2024)
- [Road Planning for Slums via Deep Reinforcement Learning](https://mlsanity.com/paper/2305.13060/) (2023)
- [ArchGym: An Open-Source Gymnasium for Machine Learning Assisted Architecture Design](https://mlsanity.com/paper/2306.08888/) (2023)
- [CoDreamer: Communication-Based Decentralised World Models](https://mlsanity.com/paper/2406.13600/) (2024)
- [X-RLflow: Graph Reinforcement Learning for Neural Network Subgraphs Transformation](https://mlsanity.com/paper/2304.14698/) (2023)
- [Reinforcement Learning for Photonic Component Design](https://mlsanity.com/paper/2307.11075/) (2023)
- [OpenRL: A Unified Reinforcement Learning Framework](https://mlsanity.com/paper/2312.16189/) (2023)
- [RL-MUL 2.0: Multiplier Design Optimization with Parallel Deep Reinforcement Learning and Space Reduction](https://mlsanity.com/paper/2404.00639/) (2024)
- [Towards a Robust Soft Baby Robot With Rich Interaction Ability for Advanced Machine Learning Algorithms](https://mlsanity.com/paper/2404.08093/) (2024)
- [GraPhSyM: Graph Physical Synthesis Model](https://mlsanity.com/paper/2308.03944/) (2023)
- [Scenario-based Thermal Management Parametrization Through Deep Reinforcement Learning](https://mlsanity.com/paper/2408.02022/) (2024)
- [QiMeng: Fully Automated Hardware and Software Design for Processor Chip](https://mlsanity.com/paper/2506.05007/) (2025)
- [AC-Refiner: Efficient Arithmetic Circuit Optimization Using Conditional Diffusion Models](https://mlsanity.com/paper/2507.02598/) (2025)
- [PrefixAgent: An LLM-Powered Design Framework for Efficient Prefix Adder Optimization](https://mlsanity.com/paper/2507.06127/) (2025)

## Abstract (arXiv)

In this work, we present a reinforcement learning (RL) based approach to designing parallel prefix circuits such as adders or priority encoders that are fundamental to high-performance digital design. Unlike prior methods, our approach designs solutions tabula rasa purely through learning with synthesis in the loop. We design a grid-based state-action representation and an RL environment for constructing legal prefix circuits. Deep Convolutional RL agents trained on this environment produce prefix adder circuits that Pareto-dominate existing baselines with up to 16.0% and 30.2% lower area for the same delay in the 32b and 64b settings respectively. We observe that agents trained with open-source synthesis tools and cell library can design adder circuits that achieve lower area and delay than commercial tool adders in an industrial cell library.