# How Can Reinforcement Learning Achieve Expert-level Placement?

arXiv: https://arxiv.org/abs/2604.25191 · HTML: https://mlsanity.com/paper/2604.25191/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 33 references that are in the map.
- First posted on arXiv 28 Apr 2026.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 8 of 8)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [IQ-Learn: Inverse soft-Q Learning for Imitation](https://mlsanity.com/paper/2106.12142/) (2021)
- [On Joint Learning for Solving Placement and Routing in Chip Design](https://mlsanity.com/paper/2111.00234/) (2021)
- [Hier-RTLMP: A Hierarchical Automatic Macro Placer for Large-scale Complex IP Blocks](https://mlsanity.com/paper/2304.11761/) (2023)
- [That Chip Has Sailed: A Critique of Unfounded Skepticism Around AI for Chip Design](https://mlsanity.com/paper/2411.10053/) (2024)

## Abstract (arXiv)

Chip placement is a critical step in physical design. While reinforcement learning (RL)-based methods have recently emerged, their training primarily focuses on wirelength optimization, and therefore often fail to achieve expert-quality layouts. We identify the reward design as the primary cause for the performance gap with experts, and instead of formalizing intricate processes, we circumvent this by directly learning from expert layouts to derive a reward model. Our approach starts from the final expert layouts to infer step-by-step expert trajectories. Using these trajectories as demonstrations or preferences, we train a model that captures the latent implicit rewards in expert results. Experiments show that our framework can efficiently learn from even a single design and generalize well to unseen cases.