# Quantum Adaptive Self-Attention for Quantum Transformer Models

arXiv: https://arxiv.org/abs/2504.05336 · HTML: https://mlsanity.com/paper/2504.05336/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 5 Apr 2025.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [Quantum Long Short-Term Memory](https://mlsanity.com/paper/2009.01783/) (2020)
- [Quantum Vision Transformers](https://mlsanity.com/paper/2209.08167/) (2022)
- [The Fine-Grained Complexity of Gradient Computation for Training Large Language Models](https://mlsanity.com/paper/2402.04497/) (2024)
- [QEEGNet: Quantum Machine Learning for Enhanced Electroencephalography Encoding](https://mlsanity.com/paper/2407.19214/) (2024)
- [Mind's Eye: Image Recognition by EEG via Multimodal Similarity-Keeping Contrastive Learning](https://mlsanity.com/paper/2406.16910/) (2024)
- [Quantum Multimodal Contrastive Learning Framework](https://mlsanity.com/paper/2408.13919/) (2024)
- [NECOMIMI: Neural-Cognitive Multimodal EEG-informed Image Generation with Diffusion Models](https://mlsanity.com/paper/2410.00712/) (2024)
- [Large Cognition Model: Towards Pretrained EEG Foundation Model](https://mlsanity.com/paper/2502.17464/) (2025)
- [Path sampling of recurrent neural networks by incorporating known physics](https://mlsanity.com/paper/2203.00597/) (2022)
- [Psycho Gundam: Electroencephalography based real-time robotic control system with deep learning](https://mlsanity.com/paper/2411.06414/) (2024)
- [Exploring the Potential of QEEGNet for Cross-Task and Cross-Dataset Electroencephalography Encoding with Quantum Machine Learning](https://mlsanity.com/paper/2503.00080/) (2025)
- [Optimizing Supply Chain Networks with the Power of Graph Neural Networks](https://mlsanity.com/paper/2501.06221/) (2025)

## Built on by (most cited first, 14 of 14)

- [Quantum Reinforcement Learning Trading Agent for Sector Rotation in the Taiwan Stock Market](https://mlsanity.com/paper/2506.20930/) (2025)
- [Benchmarking Quantum and Classical Sequential Models for Urban Telecommunication Forecasting](https://mlsanity.com/paper/2508.04488/) (2025)
- [Quantum-Enhanced Natural Language Generation: A Multi-Model Framework with Hybrid Quantum-Classical Architectures](https://mlsanity.com/paper/2508.21332/) (2025)
- [Quantum Adaptive Self-Attention for Financial Rebalancing: An Empirical Study on Automated Market Makers in Decentralized Finance](https://mlsanity.com/paper/2509.16955/) (2025)
- [Self-Modulating Quantum Fast-Weight Programmers for Efficient Adaptive Sequential Learning](https://mlsanity.com/paper/2606.24933/) (2026)
- [Vision-QRWKV: Exploring Quantum-Enhanced RWKV Models for Image Classification](https://mlsanity.com/paper/2506.06633/) (2025)
- [Q-DPTS: Quantum Differentially Private Time Series Forecasting via Variational Quantum Circuits](https://mlsanity.com/paper/2508.05036/) (2025)
- [Quantum Temporal Convolutional Neural Networks for Cross-Sectional Equity Return Prediction: A Comparative Benchmark Study](https://mlsanity.com/paper/2512.06630/) (2025)
- [Do Quantum Transformers Help? A Systematic VQC Architecture Comparison on Tabular Benchmarks](https://mlsanity.com/paper/2604.23931/) (2026)
- [Vectorized Attention with Learnable Encoding for Quantum Transformer](https://mlsanity.com/paper/2508.18464/) (2025)
- [Quantum and Classical Machine Learning in Decentralized Finance: Comparative Evidence from Multi-Asset Backtesting of Automated Market Makers](https://mlsanity.com/paper/2510.15903/) (2025)
- [QSTAformer: A Quantum-Enhanced Transformer for Robust Short-Term Voltage Stability Assessment against Adversarial Attacks](https://mlsanity.com/paper/2512.09936/) (2025)
- [A Quantum-Classical Hybrid Framework for Multivariate Time-Series Forecasting Complexity-Fidelity Trade-offs and Limitations](https://mlsanity.com/paper/2607.16358/) (2026)
- [Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models](https://mlsanity.com/paper/2609.24657/) (2026)

## Abstract (arXiv)

A recurring weakness in quantum machine learning (QML) is that reported ``quantum advantages'' are seldom tested against a capacity-matched classical control, leaving it unclear whether a gain comes from the quantum substrate or from the architectural change that accompanies it. Our primary contribution is methodological: a protocol for attributing such gains honestly -- a capacity-matched classical bottleneck of identical parameter budget, transparent reporting of where quantum does not help, and validation on real quantum hardware -- which we develop and apply through a concrete case study. That case study is Quantum Adaptive Self-Attention (QASA), a hybrid Transformer that replaces the value projection of a single encoder layer with a 36-parameter parameterized quantum circuit (PQC), keeping all other layers classical. Across nine synthetic benchmarks and the real-world ETTh1 dataset, QASA improves on a full-capacity classical Transformer for chaotic and trend-dominated signals. To ask whether this is a genuinely quantum effect, we introduce a control rarely applied in quantum machine learning -- a capacity-matched classical bottleneck with the same parameter budget -- and find that it matches the PQC on the error metrics. The gain is therefore attributable to the low-rank value-projection bottleneck (an architectural parsimony principle), not to quantumness; adding further quantum layers only degrades performance and trainability. We accordingly position the quantum layer not as a source of accuracy advantage but as a competitive instantiation of this principle: its low-rank compression onto the signal's intrinsic dimensionality is matched by a classical bottleneck, so the gain is architectural rather than quantum.