# A compact butterfly-style silicon photonic-electronic neural chip for hardware-efficient deep learning

arXiv: https://arxiv.org/abs/2111.06705 · HTML: https://mlsanity.com/paper/2111.06705/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 75 references that are in the map.
- First posted on arXiv 11 Nov 2021.

## Builds on (most cited first, 22 of 22)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation](https://mlsanity.com/paper/1404.0736/) (2014)
- [EIE: Efficient Inference Engine on Compressed Deep Neural Network](https://mlsanity.com/paper/1602.01528/) (2016)
- [cuDNN: Efficient Primitives for Deep Learning](https://mlsanity.com/paper/1410.0759/) (2014)
- [Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition](https://mlsanity.com/paper/1412.6553/) (2014)
- [All-Optical Machine Learning Using Diffractive Deep Neural Networks](https://mlsanity.com/paper/1804.08711/) (2018)
- [Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems](https://mlsanity.com/paper/2104.13386/) (2021)
- [Photonics for artificial intelligence and neuromorphic computing](https://mlsanity.com/paper/2011.00111/) (2020)
- [Convolutional neural networks with low-rank regularization](https://mlsanity.com/paper/1511.06067/) (2015)
- [Neuromorphic Silicon Photonic Networks](https://mlsanity.com/paper/1611.02272/) (2016)
- [Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit](https://mlsanity.com/paper/2008.11659/) (2020)
- [CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices](https://mlsanity.com/paper/1708.08917/) (2017)
- [Training of photonic neural networks through in situ backpropagation](https://mlsanity.com/paper/1805.09943/) (2018)
- [Efficient On-Chip Learning for Optical Neural Networks Through Power-Aware Sparse Zeroth-Order Optimization](https://mlsanity.com/paper/2012.11148/) (2020)
- [Massively Parallel Amplitude-Only Fourier Neural Network](https://mlsanity.com/paper/2008.05853/) (2020)
- [Reprogrammable Electro-Optic Nonlinear Activation Functions for Optical Neural Networks](https://mlsanity.com/paper/1903.04579/) (2019)
- [Photonic tensor cores for machine learning](https://mlsanity.com/paper/2002.03780/) (2020)
- [Building Efficient Deep Neural Networks with Unitary Group Convolutions](https://mlsanity.com/paper/1811.07755/) (2018)
- [Silicon photonic-electronic neural network for fibre nonlinearity compensation](https://mlsanity.com/paper/2110.12833/) (2021)
- [Efficient training and design of photonic neural network through neuroevolution](https://mlsanity.com/paper/1908.08012/) (2019)
- [Modeling Silicon-Photonic Neural Networks under Uncertainties](https://mlsanity.com/paper/2012.10594/) (2020)
- [Efficient Recurrent Neural Networks using Structured Matrices in FPGAs](https://mlsanity.com/paper/1803.07661/) (2018)

## Built on by (most cited first, 13 of 13)

- [NeurOLight: A Physics-Agnostic Neural Operator Enabling Parametric Photonic Device Simulation](https://mlsanity.com/paper/2209.10098/) (2022)
- [SimPhony: A Device-Circuit-Architecture Cross-Layer Modeling and Simulation Framework for Heterogeneous Electronic-Photonic AI System](https://mlsanity.com/paper/2411.13715/) (2024)
- [TeMPO: Efficient Time-Multiplexed Dynamic Photonic Tensor Core for Edge AI with Compact Slow-Light Electro-Optic Modulator](https://mlsanity.com/paper/2402.07393/) (2024)
- [SCATTER: Algorithm-Circuit Co-Sparse Photonic Accelerator with Thermal-Tolerant, Power-Efficient In-situ Light Redistribution](https://mlsanity.com/paper/2407.05510/) (2024)
- [M3ICRO: Machine Learning-Enabled Compact Photonic Tensor Core based on PRogrammable Multi-Operand Multimode Interference](https://mlsanity.com/paper/2305.19505/) (2023)
- [PACE: Pacing Operator Learning to Accurate Optical Field Simulation for Complicated Photonic Devices](https://mlsanity.com/paper/2411.03527/) (2024)
- [Integrated multi-operand optical neurons for scalable and hardware-efficient deep learning](https://mlsanity.com/paper/2305.19592/) (2023)
- [Neural Network Methods for Radiation Detectors and Imaging](https://mlsanity.com/paper/2311.05726/) (2023)
- [Hardware-Efficient Photonic Tensor Core: Accelerating Deep Neural Networks with Structured Compression](https://mlsanity.com/paper/2502.01670/) (2025)
- [DOCTOR: Dynamic On-Chip Temporal Variation Remediation Toward Self-Corrected Photonic Tensor Accelerators](https://mlsanity.com/paper/2403.02688/) (2024)
- [Efficient training for large-scale optical neural network using an evolutionary strategy and attention pruning](https://mlsanity.com/paper/2505.12906/) (2025)
- [Toward Large-Scale Photonics-Empowered AI Systems: From Physical Design Automation to System-Algorithm Co-Exploration](https://mlsanity.com/paper/2601.00129/) (2025)
- [Harnessing Photonics for Machine Intelligence](https://mlsanity.com/paper/2604.10841/) (2026)

## Abstract (arXiv)

The optical neural network (ONN) is a promising hardware platform for next-generation neurocomputing due to its high parallelism, low latency, and low energy consumption. Previous ONN architectures are mainly designed for general matrix multiplication (GEMM), leading to unnecessarily large area cost and high control complexity. Here, we move beyond classical GEMM-based ONNs and propose an optical subspace neural network (OSNN) architecture, which trades the universality of weight representation for lower optical component usage, area cost, and energy consumption. We devise a butterfly-style photonic-electronic neural chip to implement our OSNN with up to 7x fewer trainable optical components compared to GEMM-based ONNs. Additionally, a hardware-aware training framework is provided to minimize the required device programming precision, lessen the chip area, and boost the noise robustness. We experimentally demonstrate the utility of our neural chip in practical image recognition tasks, showing that a measured accuracy of 94.16% can be achieved in hand-written digit recognition tasks with 3-bit weight programming precision.