# Model selection in hybrid quantum neural networks with applications to quantum transformer architectures

arXiv: https://arxiv.org/abs/2603.21749 · HTML: https://mlsanity.com/paper/2603.21749/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 52 references that are in the map.
- First posted on arXiv 23 Mar 2026.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Supervised learning with quantum enhanced feature spaces](https://mlsanity.com/paper/1804.11326/) (2018)
- [MolGAN: An implicit generative model for small molecular graphs](https://mlsanity.com/paper/1805.11973/) (2018)
- [Quantum generative adversarial networks](https://mlsanity.com/paper/1804.08641/) (2018)
- [The Born Supremacy: Quantum Advantage and Training of an Ising Born Machine](https://mlsanity.com/paper/1904.02214/) (2019)
- [Quantum Vision Transformers](https://mlsanity.com/paper/2209.08167/) (2022)
- [Quantum Self-Attention Neural Networks for Text Classification](https://mlsanity.com/paper/2205.05625/) (2022)
- [Recent Advances for Quantum Neural Networks in Generative Learning](https://mlsanity.com/paper/2206.03066/) (2022)
- [Quantum Generative Models for Small Molecule Drug Discovery](https://mlsanity.com/paper/2101.03438/) (2021)
- [Generative Artificial Intelligence: A Systematic Review and Applications](https://mlsanity.com/paper/2405.11029/) (2024)
- [A comprehensive review of Quantum Machine Learning: from NISQ to Fault Tolerance](https://mlsanity.com/paper/2401.11351/) (2024)
- [Quantum Mixed-State Self-Attention Network](https://mlsanity.com/paper/2403.02871/) (2024)
- [Quantum Generative Diffusion Model: A Fully Quantum-Mechanical Model for Generating Quantum State Ensemble](https://mlsanity.com/paper/2401.07039/) (2024)
- [A Survey on State-of-the-art Deep Learning Applications and Challenges](https://mlsanity.com/paper/2403.17561/) (2024)
- [Learning with SASQuaTCh: a Novel Variational Quantum Transformer Architecture with Kernel-Based Self-Attention](https://mlsanity.com/paper/2403.14753/) (2024)
- [Do Quantum Neural Networks have Simplicity Bias?](https://mlsanity.com/paper/2407.03266/) (2024)
- [Quantum-Enhanced Attention Mechanism in NLP: A Hybrid Classical-Quantum Approach](https://mlsanity.com/paper/2501.15630/) (2025)
- [Enhancing variational quantum algorithms by balancing training on classical and quantum hardware](https://mlsanity.com/paper/2503.16361/) (2025)
- [Meta-learning of Gibbs states for many-body Hamiltonians with applications to Quantum Boltzmann Machines](https://mlsanity.com/paper/2507.16373/) (2025)
- [Quantum Generative Adversarial Autoencoders: Learning latent representations for quantum data generation](https://mlsanity.com/paper/2509.16186/) (2025)
- [The Bias-Expressivity Trade-off](https://mlsanity.com/paper/1911.04964/) (2019)

## Built on by (most cited first, 1 of 1)

- [AutoQuREO: A Framework for Automated Quantum Resource Estimation and Optimization](https://mlsanity.com/paper/2608.12936/) (2026)

## Abstract (arXiv)

Quantum machine learning models generally lack principled design guidelines, often requiring full resource-intensive training across numerous choices of encodings, quantum circuit designs and initialization strategies to find effective configuration. To address this challenge, we develope the Quantum Bias-Expressivity Toolbox (textttQBET), a framework for evaluating quantum, classical, and hybrid transformer architectures. In this toolbox, we introduce lean metrics for Simplicity Bias (textttSB) and Expressivity (textttEXP), for comparing across various models, and extend the analysis of textttSB to generative and multiclass-classification tasks. We show that textttQBET enables efficient pre-screening of promising model variants obviating the need to execute complete training pipelines. In evaluations on transformer-based classification and generative tasks we employ a total of 18 qubits for embeddings (6 qubits each for query, key, and value). We identify scenarios in which quantum self-attention variants surpass their classical counterparts by ranking the respective models according to the textttSB metric and comparing their relative performance.