# Quantum Multi-Agent Reinforcement Learning via Variational Quantum Circuit Design

arXiv: https://arxiv.org/abs/2203.10443 · HTML: https://mlsanity.com/paper/2203.10443/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 12 references that are in the map.
- First posted on arXiv 20 Mar 2022.

## Builds on (most cited first, 6 of 6)

- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuits](https://mlsanity.com/paper/2107.10845/) (2021)
- [Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications](https://mlsanity.com/paper/2008.02608/) (2020)
- [Quantum Deep Learning](https://mlsanity.com/paper/1412.3489/) (2014)
- [Introduction to Quantum Reinforcement Learning: Theory and PennyLane-based Implementation](https://mlsanity.com/paper/2108.06849/) (2021)
- [Quantum Neural Networks: Concepts, Applications, and Challenges](https://mlsanity.com/paper/2108.01468/) (2021)

## Built on by (most cited first, 25 of 25)

- [A Survey on Quantum Reinforcement Learning](https://mlsanity.com/paper/2211.03464/) (2022)
- [Quantum deep recurrent reinforcement learning](https://mlsanity.com/paper/2210.14876/) (2022)
- [Quantum Multi-Agent Meta Reinforcement Learning](https://mlsanity.com/paper/2208.11510/) (2022)
- [Asynchronous training of quantum reinforcement learning](https://mlsanity.com/paper/2301.05096/) (2023)
- [Learning to Measure Quantum Neural Networks](https://mlsanity.com/paper/2501.05663/) (2025)
- [Reservoir Computing via Quantum Recurrent Neural Networks](https://mlsanity.com/paper/2211.02612/) (2022)
- [Efficient quantum recurrent reinforcement learning via quantum reservoir computing](https://mlsanity.com/paper/2309.07339/) (2023)
- [eQMARL: Entangled Quantum Multi-Agent Reinforcement Learning for Distributed Cooperation over Quantum Channels](https://mlsanity.com/paper/2405.17486/) (2024)
- [Scalable Quantum Convolutional Neural Networks](https://mlsanity.com/paper/2209.12372/) (2022)
- [Quantum Multi-Agent Actor-Critic Networks for Cooperative Mobile Access in Multi-UAV Systems](https://mlsanity.com/paper/2302.04445/) (2023)
- [3D Scalable Quantum Convolutional Neural Networks for Point Cloud Data Processing in Classification Applications](https://mlsanity.com/paper/2210.09728/) (2022)
- [Projection Valued Measure-based Quantum Machine Learning for Multi-Class Classification](https://mlsanity.com/paper/2210.16731/) (2022)
- [Quantum Split Neural Network Learning using Cross-Channel Pooling](https://mlsanity.com/paper/2211.06524/) (2022)
- [Quantum Federated Learning with Entanglement Controlled Circuits and Superposition Coding](https://mlsanity.com/paper/2212.01732/) (2022)
- [Robustness and Generalization in Quantum Reinforcement Learning via Lipschitz Regularization](https://mlsanity.com/paper/2410.21117/) (2024)
- [Transfer Learning Analysis of Variational Quantum Circuits](https://mlsanity.com/paper/2501.01507/) (2025)
- [Quantum deep Q learning with distributed prioritized experience replay](https://mlsanity.com/paper/2304.09648/) (2023)
- [GPA: Grover Policy Agent for Generating Optimal Quantum Sensor Circuits](https://mlsanity.com/paper/2502.13755/) (2025)
- [Quantum-Efficient Reinforcement Learning Solutions for Last-Mile On-Demand Delivery](https://mlsanity.com/paper/2508.09183/) (2025)
- [Neural Architecture Search for Quantum Autoencoders](https://mlsanity.com/paper/2511.19246/) (2025)
- [Learning to Coordinate via Quantum Entanglement in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2602.08965/) (2026)
- [Software Simulation and Visualization of Quantum Multi-Drone Reinforcement Learning](https://mlsanity.com/paper/2211.15375/) (2022)
- [Architectural Influence on Variational Quantum Circuits in Multi-Agent Reinforcement Learning: Evolutionary Strategies for Optimization](https://mlsanity.com/paper/2407.20739/) (2024)
- [HCQA: Hybrid Classical-Quantum Agent for Generating Optimal Quantum Sensor Circuits](https://mlsanity.com/paper/2508.21246/) (2025)
- [Quantum Advantage in Multi Agent Reinforcement Learning](https://mlsanity.com/paper/2605.14235/) (2026)

## Abstract (arXiv)

In recent years, quantum computing (QC) has been getting a lot of attention from industry and academia. Especially, among various QC research topics, variational quantum circuit (VQC) enables quantum deep reinforcement learning (QRL). Many studies of QRL have shown that the QRL is superior to the classical reinforcement learning (RL) methods under the constraints of the number of training parameters. This paper extends and demonstrates the QRL to quantum multi-agent RL (QMARL). However, the extension of QRL to QMARL is not straightforward due to the challenge of the noise intermediate-scale quantum (NISQ) and the non-stationary properties in classical multi-agent RL (MARL). Therefore, this paper proposes the centralized training and decentralized execution (CTDE) QMARL framework by designing novel VQCs for the framework to cope with these issues. To corroborate the QMARL framework, this paper conducts the QMARL demonstration in a single-hop environment where edge agents offload packets to clouds. The extensive demonstration shows that the proposed QMARL framework enhances 57.7% of total reward than classical frameworks.