mlsanityData to 7 Oct 2026

Acceleration Method for Learning Fine-Layered Optical Neural Networks

arXiv 2109.01731 ↗First posted 1 Sep 2021cs.LG physics.optics quant-phcited by 0

Not on the map. Why?

Abstract (arXiv)

An optical neural network (ONN) is a promising system due to its high-speed and low-power operation. Its linear unit performs a multiplication of an input vector and a weight matrix in optical analog circuits. Among them, a circuit with a multiple-layered structure of programmable Mach-Zehnder interferometers (MZIs) can realize a specific class of unitary matrices with a limited number of MZIs as its weight matrix. The circuit is effective for balancing the number of programmable MZIs and ONN performance. However, it takes a lot of time to learn MZI parameters of the circuit with a conventional automatic differentiation (AD), which machine learning platforms are equipped with. To solve the time-consuming problem, we propose an acceleration method for learning MZI parameters. We create customized complex-valued derivatives for an MZI, exploiting Wirtinger derivatives and a chain rule. They are incorporated into our newly developed function module implemented in C++ to collectively calculate their values in a multi-layered structure. Our method is simple, fast, and versatile as well as compatible with the conventional AD. We demonstrate that our method works 20 times faster than the conventional AD when a pixel-by-pixel MNIST task is performed in a complex-valued recurrent neural network with an MZI-based hidden unit.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 33 references that are in the map. First posted on arXiv 1 Sep 2021.

Builds on

16 of 33 references.

References · 16

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2015Automatic differentiation in machine learning: a surveycited by 846
  3. 2015Unitary Evolution Recurrent Neural Networkscited by 291
  4. 2018All-Optical Machine Learning Using Diffractive Deep Neural Networkscited by 175
  5. 2015A Simple Way to Initialize Recurrent Networks of Rectified Linear Unitscited by 130
  6. 2016Full-Capacity Unitary Recurrent Neural Networkscited by 90
  7. 2017On orthogonality and learning recurrent networks with long term dependenciescited by 82
  8. 2016Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNscited by 62
  9. 2018Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?cited by 62
  10. 2016Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflectionscited by 56
  11. 2019Wave Physics as an Analog Recurrent Neural Networkcited by 42
  12. 2018Complex Gated Recurrent Neural Networkscited by 17
4 more
  1. 2018Complex Unitary Recurrent Neural Networks using Scaled Cayley Transformcited by 12
  2. 2019Design of optical neural networks with component imprecisionscited by 10
  3. 2014Fast Approximation of Rotations and Hessians matricescited by 8
  4. 2018Matrix optimization on universal unitary photonic devicescited by 5

Built on by

Not cited yet.