# Quantum Large Language Models via Tensor Network Disentanglers

arXiv: https://arxiv.org/abs/2410.17397 · HTML: https://mlsanity.com/paper/2410.17397/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 38 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Speeding up Convolutional Neural Networks with Low Rank Expansions](https://mlsanity.com/paper/1405.3866/) (2014)
- [Tensorizing Neural Networks](https://mlsanity.com/paper/1509.06569/) (2015)
- [Tensor Networks Meet Neural Networks: A Survey and Future Perspectives](https://mlsanity.com/paper/2302.09019/) (2023)
- [CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networks](https://mlsanity.com/paper/2401.14109/) (2024)
- [Quantum-Inspired Tensor Neural Networks for Partial Differential Equations](https://mlsanity.com/paper/2208.02235/) (2022)
- [Improving Gradient Methods via Coordinate Transformations: Applications to Quantum Machine Learning](https://mlsanity.com/paper/2304.06768/) (2023)
- [Language Design as Information Renormalization](https://mlsanity.com/paper/1708.01525/) (2017)

## Built on by (most cited first, 7 of 7)

- [Tensorization is a powerful but underexplored tool for compression and interpretability of neural networks](https://mlsanity.com/paper/2505.20132/) (2025)
- [Tensorization of neural networks for improved privacy and interpretability](https://mlsanity.com/paper/2501.06300/) (2025)
- [A Practical Tensor-Network Compression Pipeline for Production-Scale Large Language Models](https://mlsanity.com/paper/2602.01613/) (2026)
- [Quantum-enhanced Large Language Models on Quantum Hardware via Cayley Unitary Adapters](https://mlsanity.com/paper/2605.05914/) (2026)
- [Explaining Anomalies with Tensor Networks](https://mlsanity.com/paper/2505.03911/) (2025)
- [PEPS: Quantum-Inspired Reinforcement Learning for Coherent Reasoning Traces in LLMs](https://mlsanity.com/paper/2509.20105/) (2025)
- [Fisher Information, Training and Bias in Fourier Regression Models](https://mlsanity.com/paper/2510.06945/) (2025)

## Abstract (arXiv)

We introduce a framework for seamlessly integrating quantum computing into pretrained large language models (LLMs). The key idea is to construct a hybrid quantum-classical representation that exactly reproduces the original model, providing a principled starting point from which quantum resources can only improve performance. Our approach replaces the weight matrices in self-attention and multilayer perceptron layers with two variational quantum circuits coupled to a matrix product operator (MPO). Tensor network disentanglers transfer much of each layer's information into the quantum circuits, enabling the remaining tensor network to be compressed to a bond-dimension-one MPO with over three orders of magnitude fewer classical parameters (in our experiments, from 110,592 to approximately 36 for the replaced layer) and less than a 0.3% increase in perplexity. Training an added unitary adapter on top of this representation then surpasses the original model, reducing perplexity by up to 1.6%. Finally, we validate the hybrid architecture on a real quantum processor, demonstrating a practical route towards quantum-enhanced language models.