# Chip Placement with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2004.10746 · HTML: https://mlsanity.com/paper/2004.10746/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Apr 2020.

## Built on by (most cited first, 30 of 83)

- [Is Conditional Generative Modeling all you need for Decision-Making?](https://mlsanity.com/paper/2211.15657/) (2022)
- [RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts](https://mlsanity.com/paper/2411.15114/) (2024)
- [Sym-NCO: Leveraging Symmetricity for Neural Combinatorial Optimization](https://mlsanity.com/paper/2205.13209/) (2022)
- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [Conservative Objective Models for Effective Offline Model-Based Optimization](https://mlsanity.com/paper/2107.06882/) (2021)
- [Learning Collaborative Policies to Solve NP-hard Routing Problems](https://mlsanity.com/paper/2110.13987/) (2021)
- [Machine Learning for Electronic Design Automation: A Survey](https://mlsanity.com/paper/2102.03357/) (2021)
- [A Survey of Temporal Credit Assignment in Deep Reinforcement Learning](https://mlsanity.com/paper/2312.01072/) (2023)
- [PD-MORL: Preference-Driven Multi-Objective Reinforcement Learning Algorithm](https://mlsanity.com/paper/2208.07914/) (2022)
- [A Survey of Machine Learning for Computer Architecture and Systems](https://mlsanity.com/paper/2102.07952/) (2021)
- [Equivariant quantum circuits for learning on weighted graphs](https://mlsanity.com/paper/2205.06109/) (2022)
- [Efficient Deep Reinforcement Learning Requires Regulating Overfitting](https://mlsanity.com/paper/2304.10466/) (2023)
- [Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNets](https://mlsanity.com/paper/2412.07775/) (2024)
- [Data-Driven Offline Decision-Making via Invariant Representation Learning](https://mlsanity.com/paper/2211.11349/) (2022)
- [Global Counterfactual Explainer for Graph Neural Networks](https://mlsanity.com/paper/2210.11695/) (2022)
- [Data-Driven Offline Optimization For Architecting Hardware Accelerators](https://mlsanity.com/paper/2110.11346/) (2021)
- [MuZero with Self-competition for Rate Control in VP9 Video Compression](https://mlsanity.com/paper/2202.06626/) (2022)
- [DITTO: Offline Imitation Learning with World Models](https://mlsanity.com/paper/2302.03086/) (2023)
- [Pareto Set Learning for Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2501.06773/) (2025)
- [Large-scale graph representation learning with very deep GNNs and self-supervision](https://mlsanity.com/paper/2107.09422/) (2021)
- [PrefixRL: Optimization of Parallel Prefix Circuits using Deep Reinforcement Learning](https://mlsanity.com/paper/2205.07000/) (2022)
- [An Evolutionary Approach to Dynamic Introduction of Tasks in Large-scale Multitask Learning Systems](https://mlsanity.com/paper/2205.12755/) (2022)
- [Importance Weighted Actor-Critic for Optimal Conservative Offline Reinforcement Learning](https://mlsanity.com/paper/2301.12714/) (2023)
- [CLDG: Contrastive Learning on Dynamic Graphs](https://mlsanity.com/paper/2412.14451/) (2024)
- [muNet: Evolving Pretrained Deep Neural Networks into Scalable Auto-tuning Multitask Systems](https://mlsanity.com/paper/2205.10937/) (2022)
- [Beyond Average Return in Markov Decision Processes](https://mlsanity.com/paper/2310.20266/) (2023)
- [Failures Are Fated, But Can Be Faded: Characterizing and Mitigating Unwanted Behaviors in Large-Scale Vision and Language Models](https://mlsanity.com/paper/2406.07145/) (2024)
- [Fast and Data-Efficient Training of Rainbow: an Experimental Study on Atari](https://mlsanity.com/paper/2111.10247/) (2021)
- [DeepTPI: Test Point Insertion with Deep Reinforcement Learning](https://mlsanity.com/paper/2206.06975/) (2022)
- [Sibyl: Adaptive and Extensible Data Placement in Hybrid Storage Systems Using Online Reinforcement Learning](https://mlsanity.com/paper/2205.07394/) (2022)

## Abstract (arXiv)

In this work, we present a learning-based approach to chip placement, one of the most complex and time-consuming stages of the chip design process. Unlike prior methods, our approach has the ability to learn from past experience and improve over time. In particular, as we train over a greater number of chip blocks, our method becomes better at rapidly generating optimized placements for previously unseen chip blocks. To achieve these results, we pose placement as a Reinforcement Learning (RL) problem and train an agent to place the nodes of a chip netlist onto a chip canvas. To enable our RL policy to generalize to unseen blocks, we ground representation learning in the supervised task of predicting placement quality. By designing a neural architecture that can accurately predict reward across a wide variety of netlists and their placements, we are able to generate rich feature embeddings of the input netlists. We then use this architecture as the encoder of our policy and value networks to enable transfer learning. Our objective is to minimize PPA (power, performance, and area), and we show that, in under 6 hours, our method can generate placements that are superhuman or comparable on modern accelerator netlists, whereas existing baselines require human experts in the loop and take several weeks.