# Delving into Macro Placement with Reinforcement Learning

arXiv: https://arxiv.org/abs/2109.02587 · HTML: https://mlsanity.com/paper/2109.02587/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 7 references that are in the map.
- First posted on arXiv 6 Sep 2021.

## Built on by (most cited first, 8 of 8)

- [MaskPlace: Fast Chip Placement via Reinforced Visual Representation Learning](https://mlsanity.com/paper/2211.13382/) (2022)
- [ChiPFormer: Transferable Chip Placement via Offline Decision Transformer](https://mlsanity.com/paper/2306.14744/) (2023)
- [LLM4SecHW: Leveraging Domain Specific Large Language Model for Hardware Debugging](https://mlsanity.com/paper/2401.16448/) (2024)
- [That Chip Has Sailed: A Critique of Unfounded Skepticism Around AI for Chip Design](https://mlsanity.com/paper/2411.10053/) (2024)
- [Theta-Resonance: A Single-Step Reinforcement Learning Method for Design Space Exploration](https://mlsanity.com/paper/2211.02052/) (2022)
- [The False Dawn: Reevaluating Google's Reinforcement Learning for Chip Macro Placement](https://mlsanity.com/paper/2306.09633/) (2023)
- [CORE: Constraint-Aware One-Step Reinforcement Learning for Simulation-Guided Neural Network Accelerator Design](https://mlsanity.com/paper/2506.03474/) (2025)
- [Inverse Design in Distributed Circuits Using Single-Step Reinforcement Learning](https://mlsanity.com/paper/2506.08029/) (2025)

## Abstract (arXiv)

In physical design, human designers typically place macros via trial and error, which is a Markov decision process. Reinforcement learning (RL) methods have demonstrated superhuman performance on the macro placement. In this paper, we propose an extension to this prior work (Mirhoseini et al., 2020). We first describe the details of the policy and value network architecture. We replace the force-directed method with DREAMPlace for placing standard cells in the RL environment. We also compare our improved method with other academic placers on public benchmarks.