# Reinforcement Learning Policy as Macro Regulator Rather than Macro Placer

arXiv: https://arxiv.org/abs/2412.07167 · HTML: https://mlsanity.com/paper/2412.07167/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 7 of its 35 references that are in the map.
- First posted on arXiv 10 Dec 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 7 of 7)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [On Joint Learning for Solving Placement and Routing in Chip Design](https://mlsanity.com/paper/2111.00234/) (2021)
- [MaskPlace: Fast Chip Placement via Reinforced Visual Representation Learning](https://mlsanity.com/paper/2211.13382/) (2022)
- [ChiPFormer: Transferable Chip Placement via Offline Decision Transformer](https://mlsanity.com/paper/2306.14744/) (2023)
- [Macro Placement by Wire-Mask-Guided Black-Box Optimization](https://mlsanity.com/paper/2306.16844/) (2023)
- [Placement in Integrated Circuits using Cyclic Reinforcement Learning and Simulated Annealing](https://mlsanity.com/paper/2011.07577/) (2020)
- [Escaping Local Optima in Global Placement](https://mlsanity.com/paper/2402.18311/) (2024)

## Built on by (most cited first, 10 of 10)

- [BBOPlace-Bench: Benchmarking Black-Box Optimization for Chip Placement](https://mlsanity.com/paper/2510.23472/) (2025)
- [Open3DBench: Open-Source Benchmark for 3D-IC Backend Implementation and PPA Evaluation](https://mlsanity.com/paper/2503.12946/) (2025)
- [See it to Place it: Evolving Macro Placements with Vision-Language Models](https://mlsanity.com/paper/2603.28733/) (2026)
- [Physics-Guided Geometric Diffusion for Macro Placement Generation](https://mlsanity.com/paper/2605.16451/) (2026)
- [Order Matters: Unveiling the Hidden Impact of Macro Placement Sequences via Proxy-Guided LLM Evolution](https://mlsanity.com/paper/2606.08904/) (2026)
- [Re2MaP: Macro Placement by Recursively Prototyping and Packing Tree-based Relocating](https://mlsanity.com/paper/2511.08054/) (2025)
- [PhysEDA: Physics-Aware Learning Framework for Efficient EDA With Manhattan Distance Decay](https://mlsanity.com/paper/2605.10547/) (2026)
- [EDATracer: An Agentic Framework for Large-Scale EDA Artifact Analysis](https://mlsanity.com/paper/2608.04032/) (2026)
- [PPAPlace: Differentiable Cross-Stage Objectives for Chip Placement Optimization](https://mlsanity.com/paper/2608.13790/) (2026)
- [MacroAgent: Regularity-Aware Macro Legalization with LLM-Agent-Designed Contour Algorithms](https://mlsanity.com/paper/2608.24946/) (2026)

## Abstract (arXiv)

In modern chip design, placement aims at placing millions of circuit modules, which is an essential step that significantly influences power, performance, and area (PPA) metrics. Recently, reinforcement learning (RL) has emerged as a promising technique for improving placement quality, especially macro placement. However, current RL-based placement methods suffer from long training times, low generalization ability, and inability to guarantee PPA results. A key issue lies in the problem formulation, i.e., using RL to place from scratch, which results in limits useful information and inaccurate rewards during the training process. In this work, we propose an approach that utilizes RL for the refinement stage, which allows the RL policy to learn how to adjust existing placement layouts, thereby receiving sufficient information for the policy to act and obtain relatively dense and precise rewards. Additionally, we introduce the concept of regularity during training, which is considered an important metric in the chip design industry but is often overlooked in current RL placement methods. We evaluate our approach on the ISPD 2005 and ICCAD 2015 benchmark, comparing the global half-perimeter wirelength and regularity of our proposed method against several competitive approaches. Besides, we test the PPA performance using commercial software, showing that RL as a regulator can achieve significant PPA improvements. Our RL regulator can fine-tune placements from any method and enhance their quality. Our work opens up new possibilities for the application of RL in placement, providing a more effective and efficient approach to optimizing chip design. Our code is available at urlhttps://github.com/lamda-bbo/macro-regulator.