# ChiPFormer: Transferable Chip Placement via Offline Decision Transformer

arXiv: https://arxiv.org/abs/2306.14744 · HTML: https://mlsanity.com/paper/2306.14744/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 28 of its 70 references that are in the map.
- First posted on arXiv 26 Jun 2023.

## Placed in

- Offline RL and imitation learning from data / [Decision transformers and sequence modeling](https://mlsanity.com/offlinerl/decision-transformers-and-sequence-modeling/)

## Builds on (most cited first, 28 of 28)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemble](https://mlsanity.com/paper/2107.00591/) (2021)
- [Online Decision Transformer](https://mlsanity.com/paper/2202.05607/) (2022)
- [RvS: What is Essential for Offline RL via Supervised Learning?](https://mlsanity.com/paper/2112.10751/) (2021)
- [Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learning](https://mlsanity.com/paper/2106.04895/) (2021)
- [Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills](https://mlsanity.com/paper/2104.07749/) (2021)
- [Generalized Decision Transformer for Offline Hindsight Information Matching](https://mlsanity.com/paper/2111.10364/) (2021)
- [Learning Cut Selection for Mixed-Integer Linear Programming via Hierarchical Sequence Model](https://mlsanity.com/paper/2302.00244/) (2023)
- [On Joint Learning for Solving Placement and Routing in Chip Design](https://mlsanity.com/paper/2111.00234/) (2021)
- [MaskPlace: Fast Chip Placement via Reinforced Visual Representation Learning](https://mlsanity.com/paper/2211.13382/) (2022)
- [Behavior Proximal Policy Optimization](https://mlsanity.com/paper/2302.11312/) (2023)
- [DARA: Dynamics-Aware Reward Augmentation in Offline Reinforcement Learning](https://mlsanity.com/paper/2203.06662/) (2022)
- [Beyond Reward: Offline Preference-guided Policy Optimization](https://mlsanity.com/paper/2305.16217/) (2023)
- [Sample-Efficient Reinforcement Learning via Conservative Model-Based Actor-Critic](https://mlsanity.com/paper/2112.10504/) (2021)
- [The Cost of Application-Class Processing: Energy and Performance Analysis of a Linux-ready 1.7GHz 64bit RISC-V Core in 22nm FDSOI Technology](https://mlsanity.com/paper/1904.05442/) (2019)
- [Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learning](https://mlsanity.com/paper/2104.05043/) (2021)
- [Offline Preference-Based Apprenticeship Learning](https://mlsanity.com/paper/2107.09251/) (2021)
- [Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learning](https://mlsanity.com/paper/2110.12997/) (2021)
- [Placement in Integrated Circuits using Cyclic Reinforcement Learning and Simulated Annealing](https://mlsanity.com/paper/2011.07577/) (2020)
- [Efficient Exploration in Resource-Restricted Reinforcement Learning](https://mlsanity.com/paper/2212.06988/) (2022)
- [Delving into Macro Placement with Reinforcement Learning](https://mlsanity.com/paper/2109.02587/) (2021)

## Built on by (most cited first, 30 of 42)

- [AnalogCoder: Analog Circuit Design via Training-Free Code Generation](https://mlsanity.com/paper/2405.14918/) (2024)
- [Macro Placement by Wire-Mask-Guided Black-Box Optimization](https://mlsanity.com/paper/2306.16844/) (2023)
- [AnalogXpert: Automating Analog Topology Synthesis by Incorporating Circuit Design Expertise into Large Language Models](https://mlsanity.com/paper/2412.19824/) (2024)
- [Benchmarking End-To-End Performance of AI-Based Chip Placement Algorithms](https://mlsanity.com/paper/2407.15026/) (2024)
- [LLM-Enhanced Bayesian Optimization for Efficient Analog Layout Constraint Generation](https://mlsanity.com/paper/2406.05250/) (2024)
- [Retrieval-Guided Reinforcement Learning for Boolean Circuit Minimization](https://mlsanity.com/paper/2401.12205/) (2024)
- [Hardware Phi-1.5B: A Large Language Model Encodes Hardware Domain Specific Knowledge](https://mlsanity.com/paper/2402.01728/) (2024)
- [Reinforcement Learning Policy as Macro Regulator Rather than Macro Placer](https://mlsanity.com/paper/2412.07167/) (2024)
- [Scalable and Effective Arithmetic Tree Generation for Adder and Multiplier Designs](https://mlsanity.com/paper/2405.06758/) (2024)
- [INSIGHT: Universal Neural Simulator for Analog Circuits Harnessing Autoregressive Transformers](https://mlsanity.com/paper/2407.07346/) (2024)
- [Offline Imitation Learning with Variational Counterfactual Reasoning](https://mlsanity.com/paper/2310.04706/) (2023)
- [A Circuit Domain Generalization Framework for Efficient Logic Synthesis in Chip Design](https://mlsanity.com/paper/2309.03208/) (2023)
- [Evolution of Optimization Algorithms for Global Placement via Large Language Models](https://mlsanity.com/paper/2504.17801/) (2025)
- [PreRoutGNN for Timing Prediction with Order Preserving Partition: Global Circuit Pre-training, Local Delay Learning and Attentional Cell Modeling](https://mlsanity.com/paper/2403.00012/) (2024)
- [That Chip Has Sailed: A Critique of Unfounded Skepticism Around AI for Chip Design](https://mlsanity.com/paper/2411.10053/) (2024)
- [Effective Analog ICs Floorplanning with Relational Graph Neural Networks and Reinforcement Learning](https://mlsanity.com/paper/2411.15212/) (2024)
- [BBOPlace-Bench: Benchmarking Black-Box Optimization for Chip Placement](https://mlsanity.com/paper/2510.23472/) (2025)
- [DIDI: Diffusion-Guided Diversity for Offline Behavioral Generation](https://mlsanity.com/paper/2405.14790/) (2024)
- [FloorSet -- a VLSI Floorplanning Dataset with Design Constraints of Real-World SoCs](https://mlsanity.com/paper/2405.05480/) (2024)
- [CODMAS: A Dialectic Multi-Agent Collaborative Framework for Structured RTL Optimization](https://mlsanity.com/paper/2603.17204/) (2026)
- [RoutePlacer: An End-to-End Routability-Aware Placer with Graph Neural Network](https://mlsanity.com/paper/2406.02651/) (2024)
- [Decision Transformer for Enhancing Neural Local Search on the Job Shop Scheduling Problem](https://mlsanity.com/paper/2409.02697/) (2024)
- [See it to Place it: Evolving Macro Placements with Vision-Language Models](https://mlsanity.com/paper/2603.28733/) (2026)
- [FlowPlace: Flow Matching for Chip Placement](https://mlsanity.com/paper/2604.23658/) (2026)
- [Physics-Guided Geometric Diffusion for Macro Placement Generation](https://mlsanity.com/paper/2605.16451/) (2026)
- [FPGA Divide-and-Conquer Placement using Deep Reinforcement Learning](https://mlsanity.com/paper/2404.13061/) (2024)
- [Addressing Large Action Spaces in 3D Floorplanning via Spatial Generalization](https://mlsanity.com/paper/2406.10538/) (2024)
- [P-MOSS: Scheduling Main-Memory Indexes Over NUMA Servers Using Next Token Prediction](https://mlsanity.com/paper/2411.02933/) (2024)
- [TransPlace: Transferable Circuit Global Placement via Graph Neural Network](https://mlsanity.com/paper/2501.05667/) (2025)
- [GNN-DT: Graph Neural Network Enhanced Decision Transformer for Efficient Optimization in Dynamic Environments](https://mlsanity.com/paper/2502.01778/) (2025)

## Abstract (arXiv)

Placement is a critical step in modern chip design, aiming to determine the positions of circuit modules on the chip canvas. Recent works have shown that reinforcement learning (RL) can improve human performance in chip placement. However, such an RL-based approach suffers from long training time and low transfer ability in unseen chip circuits. To resolve these challenges, we cast the chip placement as an offline RL formulation and present ChiPFormer that enables learning a transferable placement policy from fixed offline data. ChiPFormer has several advantages that prior arts do not have. First, ChiPFormer can exploit offline placement designs to learn transferable policies more efficiently in a multi-task setting. Second, ChiPFormer can promote effective finetuning for unseen chip circuits, reducing the placement runtime from hours to minutes. Third, extensive experiments on 32 chip circuits demonstrate that ChiPFormer achieves significantly better placement quality while reducing the runtime by 10x compared to recent state-of-the-art approaches in both public benchmarks and realistic industrial tasks. The deliverables are released at https://sites.google.com/view/chipformer/home.