# Vectorized Attention with Learnable Encoding for Quantum Transformer

arXiv: https://arxiv.org/abs/2508.18464 · HTML: https://mlsanity.com/paper/2508.18464/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 31 references that are in the map.
- First posted on arXiv 25 Aug 2025.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Attention-based Quantum Tomography](https://mlsanity.com/paper/2006.12469/) (2020)
- [Beyond Classical Attention: Quantum Attention for Scalable Computation](https://mlsanity.com/paper/2307.08045/) (2023)
- [Quantum Adaptive Self-Attention for Quantum Transformer Models](https://mlsanity.com/paper/2504.05336/) (2025)
- [QReLU and m-QReLU: Two novel quantum activation functions to aid medical diagnostics](https://mlsanity.com/paper/2010.08031/) (2020)
- [Learning with SASQuaTCh: a Novel Variational Quantum Transformer Architecture with Kernel-Based Self-Attention](https://mlsanity.com/paper/2403.14753/) (2024)
- [Quantum pixel representations and compression for N-dimensional images](https://mlsanity.com/paper/2110.04405/) (2021)
- [QMIO: A tightly integrated hybrid HPCQC system](https://mlsanity.com/paper/2505.19267/) (2025)

## Abstract (arXiv)

Vectorized quantum block encoding provides a way to embed classical data into Hilbert space, offering a pathway for quantum models, such as Quantum Transformers (QT), that replace classical self-attention with quantum circuit simulations to operate more efficiently. Current QTs rely on deep parameterized quantum circuits (PQCs), rendering them vulnerable to QPU noise, and thus hindering their practical performance. In this paper, we propose the Vectorized Quantum Transformer (VQT), a model that supports ideal masked attention matrix computation through quantum approximation simulation and efficient training via vectorized nonlinear quantum encoder, yielding shot-efficient and gradient-free quantum circuit simulation (QCS) and reduced classical sampling overhead. In addition, we demonstrate an accuracy comparison for IBM and IonQ in quantum circuit simulation and competitive results in benchmarking natural language processing tasks on IBM state-of-the-art and high-fidelity Kingston QPU. Our noise intermediate-scale quantum friendly VQT approach unlocks a novel architecture for end-to-end machine learning in quantum computing.