# Quantum Mixed-State Self-Attention Network

arXiv: https://arxiv.org/abs/2403.02871 · HTML: https://mlsanity.com/paper/2403.02871/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 83 references that are in the map.
- First posted on arXiv 5 Mar 2024.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Pre-trained Models for Natural Language Processing: A Survey](https://mlsanity.com/paper/2003.08271/) (2020)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Are Transformers universal approximators of sequence-to-sequence functions?](https://mlsanity.com/paper/1912.10077/) (2019)
- [KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation](https://mlsanity.com/paper/1911.06136/) (2019)
- [The effect of data encoding on the expressive power of variational quantum machine learning models](https://mlsanity.com/paper/2008.08605/) (2020)
- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [Theoretical Limitations of Self-Attention in Neural Sequence Models](https://mlsanity.com/paper/1906.06755/) (2019)
- [A rigorous and robust quantum speed-up in supervised machine learning](https://mlsanity.com/paper/2010.02174/) (2020)
- [A Survey on Model Compression for Large Language Models](https://mlsanity.com/paper/2308.07633/) (2023)
- [Rethinking and Improving Relative Position Encoding for Vision Transformer](https://mlsanity.com/paper/2107.14222/) (2021)
- [Challenges and Opportunities in Quantum Machine Learning](https://mlsanity.com/paper/2303.09491/) (2023)
- [Supervised quantum machine learning models are kernel methods](https://mlsanity.com/paper/2101.11020/) (2021)
- [Quantum Long Short-Term Memory](https://mlsanity.com/paper/2009.01783/) (2020)
- [Quantum circuit architecture search for variational quantum algorithms](https://mlsanity.com/paper/2010.10217/) (2020)
- [Reinforcement learning for optimization of variational quantum circuit architectures](https://mlsanity.com/paper/2103.16089/) (2021)
- [QNLP in Practice: Running Compositional Models of Meaning on a Quantum Computer](https://mlsanity.com/paper/2102.12846/) (2021)
- [Multi-task Learning based Pre-trained Language Model for Code Completion](https://mlsanity.com/paper/2012.14631/) (2020)
- [Quantum Self-Attention Neural Networks for Text Classification](https://mlsanity.com/paper/2205.05625/) (2022)
- [QSAN: A Near-term Achievable Quantum Self-Attention Network](https://mlsanity.com/paper/2207.07563/) (2022)
- [Quantum Recurrent Neural Networks for Sequential Learning](https://mlsanity.com/paper/2302.03244/) (2023)
- [QKSAN: A Quantum Kernel Self-Attention Network](https://mlsanity.com/paper/2308.13422/) (2023)
- [Concentration of Data Encoding in Parameterized Quantum Circuits](https://mlsanity.com/paper/2206.08273/) (2022)
- [Intelligent Computing: The Latest Advances, Challenges and Future](https://mlsanity.com/paper/2211.11281/) (2022)

## Built on by (most cited first, 11 of 11)

- [A Numerical Gradient Inversion Attack in Variational Quantum Neural-Networks](https://mlsanity.com/paper/2504.12806/) (2025)
- [Quantum Circuit-Based Learning Models: Bridging Quantum Computing and Machine Learning](https://mlsanity.com/paper/2602.00048/) (2026)
- [Model selection in hybrid quantum neural networks with applications to quantum transformer architectures](https://mlsanity.com/paper/2603.21749/) (2026)
- [Multi-Scale Feature Fusion Quantum Depthwise Convolutional Neural Networks for Text Classification](https://mlsanity.com/paper/2405.13515/) (2024)
- [Quantum Complex-Valued Self-Attention Model](https://mlsanity.com/paper/2503.19002/) (2025)
- [Quantum Graph Transformer for NLP Sentiment Classification](https://mlsanity.com/paper/2506.07937/) (2025)
- [CLAQS: Compact Learnable All-Quantum Token Mixer with Shared-ansatz for Text Classification](https://mlsanity.com/paper/2510.06532/) (2025)
- [QLENS: Towards A Quantum Perspective of Language Transformers](https://mlsanity.com/paper/2510.11963/) (2025)
- [Lane-Frame Quantum Multimodal Driving Forecasts for the Trajectory of Autonomous Vehicles](https://mlsanity.com/paper/2511.17675/) (2025)
- [Quantum Phase Recognition via Quantum Attention Mechanism](https://mlsanity.com/paper/2602.00473/) (2026)
- [Q-Interference: Memory-Efficient Phase-Aware Quantum-Inspired Attention](https://mlsanity.com/paper/2608.17288/) (2026)

## Abstract (arXiv)

Attention mechanisms have revolutionized natural language processing. Combining them with quantum computing aims to further advance this technology. This paper introduces a novel Quantum Mixed-State Self-Attention Network (QMSAN) for natural language processing tasks. Our model leverages quantum computing principles to enhance the effectiveness of self-attention mechanisms. QMSAN uses a quantum attention mechanism based on mixed state, allowing for direct similarity estimation between queries and keys in the quantum domain. This approach leads to more effective attention coefficient calculations. We also propose an innovative quantum positional encoding scheme, implemented through fixed quantum gates within the circuit, improving the model's ability to capture sequence information without additional qubit resources. In numerical experiments of text classification tasks on public datasets, QMSAN outperforms Quantum Self-Attention Neural Network (QSANN). Furthermore, we demonstrate QMSAN's robustness in different quantum noise environments, highlighting its potential for near-term quantum devices.