# PGKET: A Photonic Gaussian Kernel Enhanced Transformer

arXiv: https://arxiv.org/abs/2507.19041 · HTML: https://mlsanity.com/paper/2507.19041/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 25 Jul 2025.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [Generalization in quantum machine learning from few training data](https://mlsanity.com/paper/2111.05292/) (2021)
- [Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel](https://mlsanity.com/paper/1908.11775/) (2019)
- [Challenges and Opportunities in Quantum Machine Learning](https://mlsanity.com/paper/2303.09491/) (2023)
- [Do CIFAR-10 Classifiers Generalize to CIFAR-10?](https://mlsanity.com/paper/1806.00451/) (2018)
- [SOFT: Softmax-free Transformer with Linear Complexity](https://mlsanity.com/paper/2110.11945/) (2021)
- [Transformer-Based Visual Segmentation: A Survey](https://mlsanity.com/paper/2304.09854/) (2023)
- [Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr"om Method](https://mlsanity.com/paper/2111.00035/) (2021)
- [Understanding quantum machine learning also requires rethinking generalization](https://mlsanity.com/paper/2306.13461/) (2023)
- [AAformer: Auto-Aligned Transformer for Person Re-Identification](https://mlsanity.com/paper/2104.00921/) (2021)
- [The physics of optical computing](https://mlsanity.com/paper/2308.00088/) (2023)
- [Visualizing and Understanding Patch Interactions in Vision Transformer](https://mlsanity.com/paper/2203.05922/) (2022)
- [Pruning Self-attentions into Convolutional Layers in Single Path](https://mlsanity.com/paper/2111.11802/) (2021)
- [HQViT: Hybrid Quantum Vision Transformer for Image Classification](https://mlsanity.com/paper/2504.02730/) (2025)
- [Gaussian Kernelized Self-Attention for Long Sequence Data and Its Application to CTC-based Speech Recognition](https://mlsanity.com/paper/2102.09168/) (2021)

## Abstract (arXiv)

Self-Attention Mechanisms (SAMs) enhance model performance by extracting key information but are inefficient when dealing with long sequences. To this end, a photonic Gaussian Kernel Enhanced Transformer (PGKET) is proposed, based on the Photonic Gaussian Kernel Self-Attention Mechanism (PGKSAM). The PGKSAM calculates the Photonic Gaussian Kernel Self-Attention Score (PGKSAS) using photon interferometry and superposition to process multiple inputs in parallel. Experimental results show that PGKET outperforms some state-of-the-art transformers in multi-classification tasks on MedMNIST v2 and CIFAR-10, and is expected to improve performance in complex tasks and accelerate the convergence of Photonic Computing (PC) and machine learning.