# TRON: Transformer Neural Network Acceleration with Non-Coherent Silicon Photonics

arXiv: https://arxiv.org/abs/2303.12914 · HTML: https://mlsanity.com/paper/2303.12914/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 22 Mar 2023.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Hardware Accelerator for Multi-Head Attention and Position-Wise Feed-Forward in the Transformer](https://mlsanity.com/paper/2009.08605/) (2020)
- [Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimization](https://mlsanity.com/paper/2110.10030/) (2021)
- [VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer](https://mlsanity.com/paper/2201.06618/) (2022)
- [A Survey on Silicon Photonics for Deep Learning](https://mlsanity.com/paper/2101.01751/) (2021)
- [RecLight: A Recurrent Neural Network Accelerator with Integrated Silicon Photonics](https://mlsanity.com/paper/2209.00084/) (2022)

## Built on by (most cited first, 12 of 12)

- [GHOST: A Graph Neural Network Accelerator using Silicon Photonics](https://mlsanity.com/paper/2307.01782/) (2023)
- [SafeLight: Enhancing Security in Optical Convolutional Neural Network Accelerators](https://mlsanity.com/paper/2411.16712/) (2024)
- [Accelerating Neural Networks for Large Language Models and Graph Processing with Silicon Photonics](https://mlsanity.com/paper/2401.06885/) (2024)
- [Silicon Photonic 2.5D Interposer Networks for Overcoming Communication Bottlenecks in Scale-out Machine Learning Hardware Accelerators](https://mlsanity.com/paper/2403.04189/) (2024)
- [OPIMA: Optical Processing-In-Memory for Convolutional Neural Network Acceleration](https://mlsanity.com/paper/2407.08205/) (2024)
- [ARTEMIS: A Mixed Analog-Stochastic In-DRAM Accelerator for Transformer Neural Networks](https://mlsanity.com/paper/2407.12638/) (2024)
- [PhotoGAN: Generative Adversarial Neural Network Acceleration with Silicon Photonics](https://mlsanity.com/paper/2501.13828/) (2025)
- [Optical Computing for Deep Neural Network Acceleration: Foundations, Recent Developments, and Emerging Directions](https://mlsanity.com/paper/2407.21184/) (2024)
- [Accelerating Diffusion Models for Generative AI Applications with Silicon Photonics](https://mlsanity.com/paper/2603.07626/) (2026)
- [DxPTA: An Architecture Design Space Exploration with Optical Dataflow-guided Strategy for HW/SW Co-Design of Photonic Transformer Accelerators](https://mlsanity.com/paper/2606.06515/) (2026)
- [MDTransformer: A Hardware-Software Co-Design of Mode-Division Photonic Transformer Accelerator with Inverse-Designed Coherent Crossbar](https://mlsanity.com/paper/2607.26016/) (2026)
- [Hardware Trojan Threats to Multi-Chiplet Photonic Neural Network Accelerators](https://mlsanity.com/paper/2609.06796/) (2026)

## Abstract (arXiv)

Transformer neural networks are rapidly being integrated into state-of-the-art solutions for natural language processing (NLP) and computer vision. However, the complex structure of these models creates challenges for accelerating their execution on conventional electronic platforms. We propose the first silicon photonic hardware neural network accelerator called TRON for transformer-based models such as BERT, and Vision Transformers. Our analysis demonstrates that TRON exhibits at least 14x better throughput and 8x better energy efficiency, in comparison to state-of-the-art transformer accelerators.