# Validity-Preserving Hierarchical RL for Joint Routing and Switch Placement in EDA

arXiv: https://arxiv.org/abs/2609.39749 · HTML: https://mlsanity.com/paper/2609.39749/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 32 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Builds on (most cited first, 5 of 5)

- [Multi-task Deep Reinforcement Learning with PopArt](https://mlsanity.com/paper/1809.04474/) (2018)
- [On Joint Learning for Solving Placement and Routing in Chip Design](https://mlsanity.com/paper/2111.00234/) (2021)
- [Batch size-invariance for policy optimization](https://mlsanity.com/paper/2110.00641/) (2021)
- [Macro Placement by Wire-Mask-Guided Black-Box Optimization](https://mlsanity.com/paper/2306.16844/) (2023)
- [A Deep Reinforcement Learning Approach for Global Routing](https://mlsanity.com/paper/1906.08809/) (2019)

## Abstract (arXiv)

Routing and switch placement are fundamental combinatorial optimization problems in chip design, requiring the joint optimization of routing topology and physical placement under strict structural, geometric and logical constraints. Existing approaches typically rely on carefully engineered heuristics that incorporate strong problem-specific biases to navigate the enormous space of possible designs. In this work, we introduce a hierarchical reinforcement learning framework for joint routing and switch placement at the level of logical communication routes. Starting from a minimal routing graph, our method progressively constructs increasingly expressive solutions through three coupled operations: switch expansion, switch placement, and route refinement. These operations preserve routing validity by construction, restricting exploration to feasible configurations where every communicating initiator-target pair has one assigned loop-free route. We explore the induced solution space using Gumbel Monte Carlo Tree Search, showing that neural-guided search substantially improves solution quality over non-learning optimization methods. Furthermore, pretraining across floorplans provides a strong initialization for fine-tuning on unseen instances.