# Inverse Design of Grating Couplers Using the Policy Gradient Method from Reinforcement Learning

arXiv: https://arxiv.org/abs/2107.00088 · HTML: https://mlsanity.com/paper/2107.00088/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 75 references that are in the map.
- First posted on arXiv 30 Jun 2021.

## Builds on (most cited first, 14 of 14)

- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Photonics for artificial intelligence and neuromorphic computing](https://mlsanity.com/paper/2011.00111/) (2020)
- [Deep neural networks for the evaluation and design of photonic devices](https://mlsanity.com/paper/2007.00084/) (2020)
- [Wave Physics as an Analog Recurrent Neural Network](https://mlsanity.com/paper/1904.12831/) (2019)
- [Global optimization of dielectric metasurfaces using a physics-driven neural network](https://mlsanity.com/paper/1906.04157/) (2019)
- [Active learning of deep surrogates for PDEs: Application to metasurface design](https://mlsanity.com/paper/2008.12649/) (2020)
- [Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows](https://mlsanity.com/paper/2006.02979/) (2020)
- [Inverse design of photonic crystals through automatic differentiation](https://mlsanity.com/paper/2003.00379/) (2020)
- [Multi-objective and categorical global optimization of photonic structures based on ResNet generative neural networks](https://mlsanity.com/paper/2007.12551/) (2020)
- [Migrating Knowledge between Physical Scenarios based on Artificial Neural Networks](https://mlsanity.com/paper/1809.00972/) (2018)
- [Improving Evolutionary Strategies with Generative Neural Networks](https://mlsanity.com/paper/1901.11271/) (2019)
- [Neural Generative Models for Global Optimization with Gradients](https://mlsanity.com/paper/1805.08594/) (2018)

## Built on by (most cited first, 3 of 3)

- [Hybrid Supervised and Reinforcement Learning for the Design and Optimization of Nanophotonic Structures](https://mlsanity.com/paper/2209.04447/) (2022)
- [Inverse design of the transmission matrix in a random system using Reinforcement Learning](https://mlsanity.com/paper/2506.13057/) (2025)
- [Machine-Learning-Assisted Photonic Device Development: A Multiscale Approach from Theory to Characterization](https://mlsanity.com/paper/2506.20056/) (2025)

## Abstract (arXiv)

We present a proof-of-concept technique for the inverse design of electromagnetic devices motivated by the policy gradient method in reinforcement learning, named PHORCED (PHotonic Optimization using REINFORCE Criteria for Enhanced Design). This technique uses a probabilistic generative neural network interfaced with an electromagnetic solver to assist in the design of photonic devices, such as grating couplers. We show that PHORCED obtains better performing grating coupler designs than local gradient-based inverse design via the adjoint method, while potentially providing faster convergence over competing state-of-the-art generative methods. As a further example of the benefits of this method, we implement transfer learning with PHORCED, demonstrating that a neural network trained to optimize 8^circ grating couplers can then be re-trained on grating couplers with alternate scattering angles while requiring >10× fewer simulations than control cases.