mlsanityData to 14 Oct 2026

When Every Simulation Counts: Value-Based Reinforcement Learning for Accelerated Photonics Inverse Design

arXiv 2607.23469 ↗First posted 26 Jul 2026physics.optics cs.AI cs.LGcited by 0

On the map: Deep reinforcement learning methods → Quantum reinforcement learning

Abstract (arXiv)

Photonic-crystal surface-emitting lasers (PCSELs) can combine high-power operation with narrow-divergence surface emission, but optimizing coupled parameters requires costly full-wave simulations. Deep Q-network (DQN) optimization can reuse simulated transitions to guide edits, yet which value-learning mechanisms remain reliable under tight simulation budgets is unknown. We address this gap by comparing baseline DQN and six value-based variants for a seven-variable PCSEL design under a shared objective, simulator, 83-call budget, and four matched initializations. Beyond endpoints, we analyze sample efficiency, policy behavior, and physical response to separate learning gains from favorable starts or exploratory jumps. Dueling DQN is the only variant to improve all four seeds. Relative to the first evaluated designs, its selected structures increase the mean quality factor () from to (), reduce wavelength error by 64%, and increase upward power by 47%; compared with baseline DQN, they achieve a higher mean under the same budget. Other variants yield no consistent improvement; Double DQN reproduces baseline trajectories, while Rainbow-lite shows high upside but strong seed dependence. These results identify Dueling DQN as the most reliable configuration tested for simulation-budget-limited PCSEL inverse design and provide a reproducible framework for attributing algorithmic gains in scientific optimization. The source code is publicly available at https://github.com/Longying-Wen/PCSEL-RL.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods. Builds on 7 of its 49 references that are in the map. First posted on arXiv 26 Jul 2026.

Builds on

7 of 49 references.

References · 7

  1. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  2. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  3. 2017Noisy Networks for Explorationcited by 266
  4. 2020Deep neural networks for the evaluation and design of photonic devicescited by 50
  5. 2020Inverse design of photonic crystals through automatic differentiationcited by 10
  6. 2018Benchmarking five global optimization approaches for nano-optical shape optimization and parameter reconstructioncited by 8
  7. 2024Inverse Design of Photonic Crystal Surface Emitting Lasers is a Sequence Modeling Problemcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Multi-Agent Reinforcement Learning for Inverse Design in Photonic Integrated Circuits
  2. 2026Learning Design Skills as Memory Policies for Agentic Photonic Inverse Design
  3. 2025Shaping Laser Pulses with Reinforcement Learning
  4. 2026Q-Learning for 3D Coverage in VCSEL-based Optical Wireless Systems ↗
  5. 2024Model-free reinforcement learning with noisy actions for automated experimental control in optics
  6. 2025Model-free Optical Processors using In Situ Reinforcement Learning with Proximal Policy Optimization
  7. 2026LCAP: Population-Informed Latent Chip Adaptation from Few Output Probes for Photonic Neural Networks
  8. 2024OpticGAI: Generative AI-aided Deep Reinforcement Learning for Optical Networks Optimization