# Adapting Pre-trained Language Models for Quantum Natural Language Processing

arXiv: https://arxiv.org/abs/2302.13812 · HTML: https://mlsanity.com/paper/2302.13812/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 45 references that are in the map.
- First posted on arXiv 24 Feb 2023.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [Pre-trained Models for Natural Language Processing: A Survey](https://mlsanity.com/paper/2003.08271/) (2020)
- [Unitary Evolution Recurrent Neural Networks](https://mlsanity.com/paper/1511.06464/) (2015)
- [Deep Complex Networks](https://mlsanity.com/paper/1705.09792/) (2017)
- [Transfer learning in hybrid classical-quantum neural networks](https://mlsanity.com/paper/1912.08278/) (2019)
- [Full-Capacity Unitary Recurrent Neural Networks](https://mlsanity.com/paper/1611.00035/) (2016)
- [Quantum machine learning beyond kernel methods](https://mlsanity.com/paper/2110.13162/) (2021)
- [QNLP in Practice: Running Compositional Models of Meaning on a Quantum Computer](https://mlsanity.com/paper/2102.12846/) (2021)
- [Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameters](https://mlsanity.com/paper/2102.08597/) (2021)
- [Encoding word order in complex embeddings](https://mlsanity.com/paper/1912.12333/) (2019)
- [Foundations for Near-Term Quantum Natural Language Processing](https://mlsanity.com/paper/2012.03755/) (2020)
- [lambeq: An Efficient High-Level Python Library for Quantum NLP](https://mlsanity.com/paper/2110.04236/) (2021)
- [Quantum Self-Attention Neural Networks for Text Classification](https://mlsanity.com/paper/2205.05625/) (2022)
- [Quantum Natural Language Processing on Near-Term Quantum Computers](https://mlsanity.com/paper/2005.04147/) (2020)
- [Associative Long Short-Term Memory](https://mlsanity.com/paper/1602.03032/) (2016)
- [Complex Transformer: A Framework for Modeling Complex-Valued Sequence](https://mlsanity.com/paper/1910.10202/) (2019)
- [On Complex Valued Convolutional Neural Networks](https://mlsanity.com/paper/1602.09046/) (2016)
- [Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks](https://mlsanity.com/paper/1906.04393/) (2019)
- [Quantum Algorithms for Compositional Natural Language Processing](https://mlsanity.com/paper/1608.01406/) (2016)
- [Complex Gated Recurrent Neural Networks](https://mlsanity.com/paper/1806.08267/) (2018)
- [Quantum Language Model with Entanglement Embedding for Question Answering](https://mlsanity.com/paper/2008.09943/) (2020)
- [CNM: An Interpretable Complex-valued Network for Matching](https://mlsanity.com/paper/1904.05298/) (2019)
- [Interpretable Complex-Valued Neural Networks for Privacy Protection](https://mlsanity.com/paper/1901.09546/) (2019)
- [Semantic Hilbert Space for Text Representation Learning](https://mlsanity.com/paper/1902.09802/) (2019)
- [Complex and Holographic Embeddings of Knowledge Graphs: A Comparison](https://mlsanity.com/paper/1707.01475/) (2017)

## Built on by (most cited first, 2 of 2)

- [A PennyLane-Centric Dataset to Enhance LLM-based Quantum Code Generation using RAG](https://mlsanity.com/paper/2503.02497/) (2025)
- [Quantum Transfer Learning for Acceptability Judgements](https://mlsanity.com/paper/2401.07777/) (2024)

## Abstract (arXiv)

The emerging classical-quantum transfer learning paradigm has brought a decent performance to quantum computational models in many tasks, such as computer vision, by enabling a combination of quantum models and classical pre-trained neural networks. However, using quantum computing with pre-trained models has yet to be explored in natural language processing (NLP). Due to the high linearity constraints of the underlying quantum computing infrastructures, existing Quantum NLP models are limited in performance on real tasks. We fill this gap by pre-training a sentence state with complex-valued BERT-like architecture, and adapting it to the classical-quantum transfer learning scheme for sentence classification. On quantum simulation experiments, the pre-trained representation can bring 50% to 60% increases to the capacity of end-to-end quantum models.