# PQLM -- Multilingual Decentralized Portable Quantum Language Model for Privacy Protection

arXiv: https://arxiv.org/abs/2210.03221 · HTML: https://mlsanity.com/paper/2210.03221/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 35 references that are in the map.
- First posted on arXiv 6 Oct 2022.

## Builds on (most cited first, 21 of 21)

- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [What Does it Mean for a Language Model to Preserve Privacy?](https://mlsanity.com/paper/2202.05520/) (2022)
- [Recurrent Neural Network Regularization](https://mlsanity.com/paper/1409.2329/) (2014)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [Quantum Long Short-Term Memory](https://mlsanity.com/paper/2009.01783/) (2020)
- [Federated Learning: Opportunities and Challenges](https://mlsanity.com/paper/2101.05428/) (2021)
- [Federated Quantum Machine Learning](https://mlsanity.com/paper/2103.12010/) (2021)
- [Differential Privacy and Machine Learning: a Survey and Review](https://mlsanity.com/paper/1412.7584/) (2014)
- [SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets](https://mlsanity.com/paper/2008.04277/) (2020)
- [Selective Differential Privacy for Language Modeling](https://mlsanity.com/paper/2108.12944/) (2021)
- [Decentralizing Feature Extraction with Quantum Convolutional Neural Network for Automatic Speech Recognition](https://mlsanity.com/paper/2010.13309/) (2020)
- [When BERT Meets Quantum Temporal Convolution Learning for Text Classification in Heterogeneous Computing](https://mlsanity.com/paper/2203.03550/) (2022)
- [Yuan 1.0: Large-Scale Pre-trained Language Model in Zero-Shot and Few-Shot Learning](https://mlsanity.com/paper/2110.04725/) (2021)
- [Quantum machine learning with differential privacy](https://mlsanity.com/paper/2103.06232/) (2021)
- [A Short Survey of Pre-trained Language Models for Conversational AI-A NewAge in NLP](https://mlsanity.com/paper/2104.10810/) (2021)
- [A Short Review of Ethical Challenges in Clinical Natural Language Processing](https://mlsanity.com/paper/1703.10090/) (2017)
- [Federated Quantum Natural Gradient Descent for Quantum Federated Learning](https://mlsanity.com/paper/2209.00564/) (2022)

## Built on by (most cited first, 26 of 26)

- [Learning to Program Variational Quantum Circuits with Fast Weights](https://mlsanity.com/paper/2402.17760/) (2024)
- [Federated Quantum Machine Learning with Differential Privacy](https://mlsanity.com/paper/2310.06973/) (2023)
- [When LLMs Meets Acoustic Landmarks: An Efficient Approach to Integrate Speech into Large Language Models for Depression Detection](https://mlsanity.com/paper/2402.13276/) (2024)
- [Federated Quantum Long Short-term Memory (FedQLSTM)](https://mlsanity.com/paper/2312.14309/) (2023)
- [Learning to Measure Quantum Neural Networks](https://mlsanity.com/paper/2501.05663/) (2025)
- [Efficient quantum recurrent reinforcement learning via quantum reservoir computing](https://mlsanity.com/paper/2309.07339/) (2023)
- [Differentiable Quantum Architecture Search in Asynchronous Quantum Reinforcement Learning](https://mlsanity.com/paper/2407.18202/) (2024)
- [Quantum-Train-Based Distributed Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2412.08845/) (2024)
- [Quantum Machine Learning Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2407.20147/) (2024)
- [Consensus-based Distributed Quantum Kernel Learning for Speech Recognition](https://mlsanity.com/paper/2409.05770/) (2024)
- [Quantum Gradient Class Activation Map for Model Interpretability](https://mlsanity.com/paper/2408.05899/) (2024)
- [Evolutionary Optimization for Designing Variational Quantum Circuits with High Model Capacity](https://mlsanity.com/paper/2412.12484/) (2024)
- [Quantum Federated Learning With Quantum Networks](https://mlsanity.com/paper/2310.15084/) (2023)
- [An Introduction to Quantum Reinforcement Learning (QRL)](https://mlsanity.com/paper/2409.05846/) (2024)
- [Adaptive Non-local Observable on Quantum Neural Networks](https://mlsanity.com/paper/2504.13414/) (2025)
- [Differentiable Quantum Architecture Search in Quantum-Enhanced Neural Network Parameter Generation](https://mlsanity.com/paper/2505.09653/) (2025)
- [Learning to Program Quantum Measurements for Machine Learning](https://mlsanity.com/paper/2505.13525/) (2025)
- [Quantum Long Short-term Memory with Differentiable Architecture Search](https://mlsanity.com/paper/2508.14955/) (2025)
- [Transfer Learning Analysis of Variational Quantum Circuits](https://mlsanity.com/paper/2501.01507/) (2025)
- [Introduction to Quantum Machine Learning and Quantum Architecture Search](https://mlsanity.com/paper/2504.16131/) (2025)
- [Unidirectional brain-computer interface: Artificial neural network encoding natural images to fMRI response in the visual cortex](https://mlsanity.com/paper/2309.15018/) (2023)
- [Breaking Through Barren Plateaus: Reinforcement Learning Initializations for Deep Variational Quantum Circuits](https://mlsanity.com/paper/2508.18514/) (2025)
- [Recursive QLSTM with Dynamic Variational Quantum Circuit Adaptation](https://mlsanity.com/paper/2606.24932/) (2026)
- [Self-Modulating Quantum Fast-Weight Programmers for Efficient Adaptive Sequential Learning](https://mlsanity.com/paper/2606.24933/) (2026)
- [Neural Architecture Search for Quantum Autoencoders](https://mlsanity.com/paper/2511.19246/) (2025)
- [MADQRL: Distributed Quantum Reinforcement Learning Framework for Multi-Agent Environments](https://mlsanity.com/paper/2604.11131/) (2026)

## Abstract (arXiv)

With careful manipulation, malicious agents can reverse engineer private information encoded in pre-trained language models. Security concerns motivate the development of quantum pre-training. In this work, we propose a highly Portable Quantum Language Model (PQLM) that can easily transmit information to downstream tasks on classical machines. The framework consists of a cloud PQLM built with random Variational Quantum Classifiers (VQC) and local models for downstream applications. We demonstrate the ad hoc portability of the quantum model by extracting only the word embeddings and effectively applying them to downstream tasks on classical machines. Our PQLM exhibits comparable performance to its classical counterpart on both intrinsic evaluation (loss, perplexity) and extrinsic evaluation (multilingual sentiment analysis accuracy) metrics. We also perform ablation studies on the factors affecting PQLM performance to analyze model stability. Our work establishes a theoretical foundation for a portable quantum pre-trained language model that could be trained on private data and made available for public use with privacy protection guarantees.