# Inverse Design in Distributed Circuits Using Single-Step Reinforcement Learning

arXiv: https://arxiv.org/abs/2506.08029 · HTML: https://mlsanity.com/paper/2506.08029/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 28 references that are in the map.
- First posted on arXiv 2 Jun 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 11 of 11)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Understanding the impact of entropy on policy optimization](https://mlsanity.com/paper/1811.11214/) (2018)
- [GCN-RL Circuit Designer: Transferable Transistor Sizing with Graph Neural Networks and Reinforcement Learning](https://mlsanity.com/paper/2005.00406/) (2020)
- [Chip Placement with Deep Reinforcement Learning](https://mlsanity.com/paper/2004.10746/) (2020)
- [CktGNN: Circuit Graph Neural Network for Electronic Design Automation](https://mlsanity.com/paper/2308.16406/) (2023)
- [AutoCkt: Deep Reinforcement Learning of Analog Circuit Designs](https://mlsanity.com/paper/2001.01808/) (2020)
- [Revisiting Design Choices in Proximal Policy Optimization](https://mlsanity.com/paper/2009.10897/) (2020)
- [Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows](https://mlsanity.com/paper/2006.02979/) (2020)
- [Domain Knowledge-Infused Deep Learning for Automated Analog/Radio-Frequency Circuit Parameter Optimization](https://mlsanity.com/paper/2204.12948/) (2022)
- [Delving into Macro Placement with Reinforcement Learning](https://mlsanity.com/paper/2109.02587/) (2021)
- [Theta-Resonance: A Single-Step Reinforcement Learning Method for Design Space Exploration](https://mlsanity.com/paper/2211.02052/) (2022)

## Abstract (arXiv)

The goal of inverse design in distributed circuits is to generate near-optimal designs that meet a desirable transfer function specification. Existing design exploration methods use some combination of strategies involving artificial grids, differentiable evaluation procedures, and specific template topologies. However, real-world design practices often require non-differentiable evaluation procedures, varying topologies, and near-continuous placement spaces. In this paper, we propose DCIDA, a design exploration framework that learns a near-optimal design sampling policy for a target transfer function. DCIDA decides all design factors in a compound single-step action by sampling from a set of jointly-trained conditional distributions generated by the policy. Utilizing an injective interdependent ``map", DCIDA transforms raw sampled design ``actions" into uniquely equivalent physical representations, enabling the framework to learn the conditional dependencies among joint ``raw'' design decisions. Our experiments demonstrate DCIDA's Transformer-based policy network achieves significant reductions in design error compared to state-of-the-art approaches, with significantly better fit in cases involving more complex transfer functions.