# M3ICRO: Machine Learning-Enabled Compact Photonic Tensor Core based on PRogrammable Multi-Operand Multimode Interference

arXiv: https://arxiv.org/abs/2305.19505 · HTML: https://mlsanity.com/paper/2305.19505/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 61 references that are in the map.
- First posted on arXiv 31 May 2023.

## Builds on (most cited first, 12 of 12)

- [TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings](https://mlsanity.com/paper/2304.01433/) (2023)
- [Photonics for artificial intelligence and neuromorphic computing](https://mlsanity.com/paper/2011.00111/) (2020)
- [Neuromorphic Silicon Photonic Networks](https://mlsanity.com/paper/1611.02272/) (2016)
- [Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit](https://mlsanity.com/paper/2008.11659/) (2020)
- [Experimentally realized in situ backpropagation for deep learning in nanophotonic neural networks](https://mlsanity.com/paper/2205.08501/) (2022)
- [CrossLight: A Cross-Layer Optimized Silicon Photonic Neural Network Accelerator](https://mlsanity.com/paper/2102.06960/) (2021)
- [Photonic tensor cores for machine learning](https://mlsanity.com/paper/2002.03780/) (2020)
- [An Electro-Photonic System for Accelerating Deep Neural Networks](https://mlsanity.com/paper/2109.01126/) (2021)
- [A compact butterfly-style silicon photonic-electronic neural chip for hardware-efficient deep learning](https://mlsanity.com/paper/2111.06705/) (2021)
- [RecLight: A Recurrent Neural Network Accelerator with Integrated Silicon Photonics](https://mlsanity.com/paper/2209.00084/) (2022)
- [Silicon photonic-electronic neural network for fibre nonlinearity compensation](https://mlsanity.com/paper/2110.12833/) (2021)
- [Graphcore C2 Card performance for image-based deep learning application: A Report](https://mlsanity.com/paper/2002.11670/) (2020)

## Built on by (most cited first, 5 of 5)

- [Arbitrary control over multimode wave propagation for machine learning](https://mlsanity.com/paper/2402.17750/) (2024)
- [SimPhony: A Device-Circuit-Architecture Cross-Layer Modeling and Simulation Framework for Heterogeneous Electronic-Photonic AI System](https://mlsanity.com/paper/2411.13715/) (2024)
- [PACE: Pacing Operator Learning to Accurate Optical Field Simulation for Complicated Photonic Devices](https://mlsanity.com/paper/2411.03527/) (2024)
- [DOCTOR: Dynamic On-Chip Temporal Variation Remediation Toward Self-Corrected Photonic Tensor Accelerators](https://mlsanity.com/paper/2403.02688/) (2024)
- [Harnessing Photonics for Machine Intelligence](https://mlsanity.com/paper/2604.10841/) (2026)

## Abstract (arXiv)

Photonic computing shows promise for transformative advancements in machine learning (ML) acceleration, offering ultra-fast speed, massive parallelism, and high energy efficiency. However, current photonic tensor core (PTC) designs based on standard optical components hinder scalability and compute density due to their large spatial footprint. To address this, we propose an ultra-compact PTC using customized programmable multi-operand multimode interference (MOMMI) devices, named M3ICRO. The programmable MOMMI leverages the intrinsic light propagation principle, providing a single-device programmable matrix unit beyond the conventional computing paradigm of one multiply-accumulate (MAC) operation per device. To overcome the optimization difficulty of customized devices that often requires time-consuming simulation, we apply ML for optics to predict the device behavior and enable a differentiable optimization flow. We thoroughly investigate the reconfigurability and matrix expressivity of our customized PTC, and introduce a novel block unfolding method to fully exploit the computing capabilities of a complex-valued PTC for near-universal real-valued linear transformations. Extensive evaluations demonstrate that M3ICRO achieves a 3.4-9.6x smaller footprint, 1.6-4.4x higher speed, 10.6-42x higher compute density, 3.7-12x higher system throughput, and superior noise robustness compared to state-of-the-art coherent PTC designs, while maintaining close-to-digital task accuracy across various ML benchmarks. Our code is open-sourced at https://github.com/JeremieMelo/M3ICRO-MOMMI.