# Quantum Self-Attention Neural Networks for Text Classification

arXiv: https://arxiv.org/abs/2205.05625 · HTML: https://mlsanity.com/paper/2205.05625/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 67 references that are in the map.
- First posted on arXiv 11 May 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Supervised learning with quantum enhanced feature spaces](https://mlsanity.com/paper/1804.11326/) (2018)
- [The power of quantum neural networks](https://mlsanity.com/paper/2011.00027/) (2020)
- [Power of data in quantum machine learning](https://mlsanity.com/paper/2011.01938/) (2020)
- [Parameterized quantum circuits as machine learning models](https://mlsanity.com/paper/1906.07682/) (2019)
- [A rigorous and robust quantum speed-up in supervised machine learning](https://mlsanity.com/paper/2010.02174/) (2020)
- [Generalization in quantum machine learning from few training data](https://mlsanity.com/paper/2111.05292/) (2021)
- [Quantum Long Short-Term Memory](https://mlsanity.com/paper/2009.01783/) (2020)
- [Quantum machine learning beyond kernel methods](https://mlsanity.com/paper/2110.13162/) (2021)
- [Noisy intermediate-scale quantum (NISQ) algorithms](https://mlsanity.com/paper/2101.08448/) (2021)
- [QNLP in Practice: Running Compositional Models of Meaning on a Quantum Computer](https://mlsanity.com/paper/2102.12846/) (2021)
- [The Dawn of Quantum Natural Language Processing](https://mlsanity.com/paper/2110.06510/) (2021)
- [When BERT Meets Quantum Temporal Convolution Learning for Text Classification in Heterogeneous Computing](https://mlsanity.com/paper/2203.03550/) (2022)
- [Quantum Natural Language Processing on Near-Term Quantum Computers](https://mlsanity.com/paper/2005.04147/) (2020)
- [Attention-based Quantum Tomography](https://mlsanity.com/paper/2006.12469/) (2020)
- [Quantum Algorithms for Compositional Natural Language Processing](https://mlsanity.com/paper/1608.01406/) (2016)
- [Classical-to-Quantum Transfer Learning for Spoken Command Recognition Based on Quantum Neural Networks](https://mlsanity.com/paper/2110.08689/) (2021)
- [Multi-Scale Self-Attention for Text Classification](https://mlsanity.com/paper/1912.00544/) (2019)
- [Optimal quantum dataset for learning a unitary transformation](https://mlsanity.com/paper/2203.00546/) (2022)
- [Practical distributed quantum information processing with LOCCNet](https://mlsanity.com/paper/2101.12190/) (2021)
- [A Hybrid Quantum-Classical Hamiltonian Learning Algorithm](https://mlsanity.com/paper/2103.01061/) (2021)

## Built on by (most cited first, 30 of 43)

- [Quantum Vision Transformers](https://mlsanity.com/paper/2209.08167/) (2022)
- [QSAN: A Near-term Achievable Quantum Self-Attention Network](https://mlsanity.com/paper/2207.07563/) (2022)
- [Quantum Recurrent Neural Networks for Sequential Learning](https://mlsanity.com/paper/2302.03244/) (2023)
- [QKSAN: A Quantum Kernel Self-Attention Network](https://mlsanity.com/paper/2308.13422/) (2023)
- [Quantum-Inspired Machine Learning: a Survey](https://mlsanity.com/paper/2308.11269/) (2023)
- [Quantum Mixed-State Self-Attention Network](https://mlsanity.com/paper/2403.02871/) (2024)
- [Hybrid Quantum Vision Transformers for Event Classification in High Energy Physics](https://mlsanity.com/paper/2402.00776/) (2024)
- [Quantum Vision Transformers for Quark-Gluon Classification](https://mlsanity.com/paper/2405.10284/) (2024)
- [HQViT: Hybrid Quantum Vision Transformer for Image Classification](https://mlsanity.com/paper/2504.02730/) (2025)
- [Quantum Natural Language Processing](https://mlsanity.com/paper/2403.19758/) (2024)
- [GQHAN: A Grover-inspired Quantum Hard Attention Network](https://mlsanity.com/paper/2401.14089/) (2024)
- [Learning with SASQuaTCh: a Novel Variational Quantum Transformer Architecture with Kernel-Based Self-Attention](https://mlsanity.com/paper/2403.14753/) (2024)
- [Experimentally validated quantum-secure federated learning over a multi-user quantum network](https://mlsanity.com/paper/2501.12709/) (2025)
- [Quantum Natural Language Processing: A Comprehensive Review of Models, Methods, and Applications](https://mlsanity.com/paper/2504.09909/) (2025)
- [Adapting Pre-trained Language Models for Quantum Natural Language Processing](https://mlsanity.com/paper/2302.13812/) (2023)
- [Quantum Transfer Learning for Acceptability Judgements](https://mlsanity.com/paper/2401.07777/) (2024)
- [Quantum Graph Attention Network: A Novel Quantum Multi-Head Attention Mechanism for Graph Learning](https://mlsanity.com/paper/2508.17630/) (2025)
- [Quantum Graph Attention Networks: Trainable Quantum Encoders for Inductive Graph Learning](https://mlsanity.com/paper/2509.11390/) (2025)
- [Quantum Adaptive Self-Attention for Financial Rebalancing: An Empirical Study on Automated Market Makers in Decentralized Finance](https://mlsanity.com/paper/2509.16955/) (2025)
- [Hybrid quantum-classical convolutional neural network for phytoplankton classification](https://mlsanity.com/paper/2303.03707/) (2023)
- [A Survey of Classical And Quantum Sequence Models](https://mlsanity.com/paper/2312.10242/) (2023)
- [Quantum Temporal Fusion Transformer](https://mlsanity.com/paper/2508.04048/) (2025)
- [Quantum Circuit-Based Learning Models: Bridging Quantum Computing and Machine Learning](https://mlsanity.com/paper/2602.00048/) (2026)
- [Quantum-Inspired Self-Attention in a Large Language Model](https://mlsanity.com/paper/2603.03318/) (2026)
- [Model selection in hybrid quantum neural networks with applications to quantum transformer architectures](https://mlsanity.com/paper/2603.21749/) (2026)
- [Do Quantum Transformers Help? A Systematic VQC Architecture Comparison on Tabular Benchmarks](https://mlsanity.com/paper/2604.23931/) (2026)
- [Multi-Scale Feature Fusion Quantum Depthwise Convolutional Neural Networks for Text Classification](https://mlsanity.com/paper/2405.13515/) (2024)
- [Traitement quantique des langues : 'etat de l'art](https://mlsanity.com/paper/2406.15370/) (2024)
- [Quantum Complex-Valued Self-Attention Model](https://mlsanity.com/paper/2503.19002/) (2025)
- [Multilingual Machine Translation with Quantum Encoder Decoder Attention-based Convolutional Variational Circuits](https://mlsanity.com/paper/2505.09407/) (2025)

## Abstract (arXiv)

An emerging direction of quantum computing is to establish meaningful quantum applications in various fields of artificial intelligence, including natural language processing (NLP). Although some efforts based on syntactic analysis have opened the door to research in Quantum NLP (QNLP), limitations such as heavy syntactic preprocessing and syntax-dependent network architecture make them impracticable on larger and real-world data sets. In this paper, we propose a new simple network architecture, called the quantum self-attention neural network (QSANN), which can compensate for these limitations. Specifically, we introduce the self-attention mechanism into quantum neural networks and then utilize a Gaussian projected quantum self-attention serving as a sensible quantum version of self-attention. As a result, QSANN is effective and scalable on larger data sets and has the desirable property of being implementable on near-term quantum devices. In particular, our QSANN outperforms the best existing QNLP model based on syntactic analysis as well as a simple classical self-attention neural network in numerical experiments of text classification tasks on public data sets. We further show that our method exhibits robustness to low-level quantum noises and showcases resilience to quantum neural network architectures.