# Variational Quantum Transformer Architecture for Synthetic Language Generation

arXiv: https://arxiv.org/abs/2609.18565 · HTML: https://mlsanity.com/paper/2609.18565/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 14 references that are in the map.
- First posted on arXiv 16 Sep 2026.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Parameterized quantum circuits as machine learning models](https://mlsanity.com/paper/1906.07682/) (2019)
- [QNLP in Practice: Running Compositional Models of Meaning on a Quantum Computer](https://mlsanity.com/paper/2102.12846/) (2021)
- [lambeq: An Efficient High-Level Python Library for Quantum NLP](https://mlsanity.com/paper/2110.04236/) (2021)
- [Quantum Self-Attention Neural Networks for Text Classification](https://mlsanity.com/paper/2205.05625/) (2022)
- [Quantum Natural Language Processing on Near-Term Quantum Computers](https://mlsanity.com/paper/2005.04147/) (2020)
- [Quantum Natural Language Generation on Near-Term Devices](https://mlsanity.com/paper/2211.00727/) (2022)
- [Quantum Transformer: Accelerating model inference via quantum linear algebra](https://mlsanity.com/paper/2402.16714/) (2024)

## Abstract (arXiv)

We propose a compact NISQ-compatible quantum transformer architecture for synthetic QNLP sequence modelling. The model preserves the autoregressive next-token interface of a classical transformer, but replaces attention and feed-forward sublayers with variational quantum encoder blocks, connector circuits, decoder blocks and a direct two-qubit measurement readout. Token contexts are angle-encoded into small quantum registers, processed by parallel variational heads and encoder integration circuits and conditioned through decoder ancillae to produce a distribution over a four-token vocabulary. We evaluate several architecture variants on deterministic and lexicographic grammar-generation tasks against a compact classical transformer baseline. The quantum models are trainable end-to-end and learn nontrivial grammar structure, including perfect deterministic generation in individual runs and high lexicographic validity in the strongest variant. The classical baseline remains more accurate and stable and the quantum models are sensitive to initialization. The contribution is therefore not a claim of quantum advantage, but a concrete architecture and evaluation of transformer-inspired QNLP sequence modelling under near-term quantum constraints.