# QNet: A Quantum-native Sequence Encoder Architecture

arXiv: https://arxiv.org/abs/2210.17262 · HTML: https://mlsanity.com/paper/2210.17262/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 50 references that are in the map.
- First posted on arXiv 31 Oct 2022.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Pay Attention to MLPs](https://mlsanity.com/paper/2105.08050/) (2021)
- [FNet: Mixing Tokens with Fourier Transforms](https://mlsanity.com/paper/2105.03824/) (2021)
- [Quantum advantage in learning from experiments](https://mlsanity.com/paper/2112.00778/) (2021)
- [Efficient Learning for Deep Quantum Neural Networks](https://mlsanity.com/paper/1902.10445/) (2019)
- [Quantum Long Short-Term Memory](https://mlsanity.com/paper/2009.01783/) (2020)
- [Tensor2Tensor for Neural Machine Translation](https://mlsanity.com/paper/1803.07416/) (2018)
- [Recurrent Quantum Neural Networks](https://mlsanity.com/paper/2006.14619/) (2020)
- [Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNet](https://mlsanity.com/paper/2105.02723/) (2021)
- [The Dawn of Quantum Natural Language Processing](https://mlsanity.com/paper/2110.06510/) (2021)
- [Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completion](https://mlsanity.com/paper/2108.04927/) (2021)
- [TransModality: An End2End Fusion Method with Transformer for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2009.02902/) (2020)
- [EmpTransfo: A Multi-head Transformer Architecture for Creating Empathetic Dialog Systems](https://mlsanity.com/paper/2003.02958/) (2020)
- [word2ket: Space-efficient Word Embeddings inspired by Quantum Entanglement](https://mlsanity.com/paper/1911.04975/) (2019)

## Abstract (arXiv)

This work proposes QNet, a novel sequence encoder model that entirely inferences on the quantum computer using a minimum number of qubits. Let n and d represent the length of the sequence and the embedding size, respectively. The dot-product attention mechanism requires a time complexity of O(n2 · d), while QNet has merely O(n+d) quantum circuit depth. In addition, we introduce ResQNet, a quantum-classical hybrid model composed of several QNet blocks linked by residual connections, as an isomorph Transformer Encoder. We evaluated our work on various natural language processing tasks, including text classification, rating score prediction, and named entity recognition. Our models exhibit compelling performance over classical state-of-the-art models with a thousand times fewer parameters. In summary, this work investigates the advantage of machine learning on near-term quantum computers in sequential data by experimenting with natural language processing tasks.