mlsanityData to 7 Oct 2026

Quantum Mixed-State Self-Attention Network

arXiv 2403.02871 ↗First posted 5 Mar 2024quant-ph cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Attention mechanisms have revolutionized natural language processing. Combining them with quantum computing aims to further advance this technology. This paper introduces a novel Quantum Mixed-State Self-Attention Network (QMSAN) for natural language processing tasks. Our model leverages quantum computing principles to enhance the effectiveness of self-attention mechanisms. QMSAN uses a quantum attention mechanism based on mixed state, allowing for direct similarity estimation between queries and keys in the quantum domain. This approach leads to more effective attention coefficient calculations. We also propose an innovative quantum positional encoding scheme, implemented through fixed quantum gates within the circuit, improving the model's ability to capture sequence information without additional qubit resources. In numerical experiments of text classification tasks on public datasets, QMSAN outperforms Quantum Self-Attention Neural Network (QSANN). Furthermore, we demonstrate QMSAN's robustness in different quantum noise environments, highlighting its potential for near-term quantum devices.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 30 of its 83 references that are in the map. First posted on arXiv 5 Mar 2024.

Builds on

30 of 83 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  5. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  6. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  7. 2021Transformers in Vision: A Surveycited by 1,009
  8. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  9. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  10. 2019Are Transformers universal approximators of sequence-to-sequence functions?cited by 404
  11. 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
  12. 2020The effect of data encoding on the expressive power of variational quantum machine learning modelscited by 328
18 more
  1. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  2. 2019Theoretical Limitations of Self-Attention in Neural Sequence Modelscited by 286
  3. 2020A rigorous and robust quantum speed-up in supervised machine learningcited by 270
  4. 2023A Survey on Model Compression for Large Language Modelscited by 263
  5. 2021Rethinking and Improving Relative Position Encoding for Vision Transformercited by 206
  6. 2023Challenges and Opportunities in Quantum Machine Learningcited by 202
  7. 2021Supervised quantum machine learning models are kernel methodscited by 199
  8. 2020Quantum Long Short-Term Memorycited by 118
  9. 2020Quantum circuit architecture search for variational quantum algorithmscited by 81
  10. 2021Reinforcement learning for optimization of variational quantum circuit architecturescited by 76
  11. 2021QNLP in Practice: Running Compositional Models of Meaning on a Quantum Computercited by 65
  12. 2020Multi-task Learning based Pre-trained Language Model for Code Completioncited by 58
  13. 2022Quantum Self-Attention Neural Networks for Text Classificationcited by 42
  14. 2022QSAN: A Near-term Achievable Quantum Self-Attention Networkcited by 25
  15. 2023Quantum Recurrent Neural Networks for Sequential Learningcited by 24
  16. 2023QKSAN: A Quantum Kernel Self-Attention Networkcited by 23
  17. 2022Concentration of Data Encoding in Parameterized Quantum Circuitscited by 15
  18. 2022Intelligent Computing: The Latest Advances, Challenges and Futurecited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 4.

11 papers.

  1. 2025A Numerical Gradient Inversion Attack in Variational Quantum Neural-Networkscited by 1
  2. 2026Quantum Circuit-Based Learning Models: Bridging Quantum Computing and Machine Learningcited by 1
  3. 2026Model selection in hybrid quantum neural networks with applications to quantum transformer architecturescited by 1
  4. 2024Multi-Scale Feature Fusion Quantum Depthwise Convolutional Neural Networks for Text Classificationcited by 0
  5. 2025Quantum Complex-Valued Self-Attention Modelcited by 0
  6. 2025Quantum Graph Transformer for NLP Sentiment Classificationcited by 0
  7. 2025CLAQS: Compact Learnable All-Quantum Token Mixer with Shared-ansatz for Text Classificationcited by 0
  8. 2025QLENS: Towards A Quantum Perspective of Language Transformerscited by 0
  9. 2025Lane-Frame Quantum Multimodal Driving Forecasts for the Trajectory of Autonomous Vehiclescited by 0
  10. 2026Quantum Phase Recognition via Quantum Attention Mechanismcited by 0
  11. 2026Q-Interference: Memory-Efficient Phase-Aware Quantum-Inspired Attentioncited by 0