# Architectural Influence on Variational Quantum Circuits in Multi-Agent Reinforcement Learning: Evolutionary Strategies for Optimization

arXiv: https://arxiv.org/abs/2407.20739 · HTML: https://mlsanity.com/paper/2407.20739/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 36 references that are in the map.
- First posted on arXiv 30 Jul 2024.

## Builds on (most cited first, 14 of 14)

- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Agent57: Outperforming the Atari Human Benchmark](https://mlsanity.com/paper/2003.13350/) (2020)
- [Learning with Opponent-Learning Awareness](https://mlsanity.com/paper/1709.04326/) (2017)
- [Multi-agent Reinforcement Learning in Sequential Social Dilemmas](https://mlsanity.com/paper/1702.03037/) (2017)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity](https://mlsanity.com/paper/1707.09183/) (2017)
- [Variational Quantum Reinforcement Learning via Evolutionary Optimization](https://mlsanity.com/paper/2109.00540/) (2021)
- [Maintaining cooperation in complex social dilemmas using deep reinforcement learning](https://mlsanity.com/paper/1707.01068/) (2017)
- [Quantum Multi-Agent Reinforcement Learning via Variational Quantum Circuit Design](https://mlsanity.com/paper/2203.10443/) (2022)
- [Quantum deep recurrent reinforcement learning](https://mlsanity.com/paper/2210.14876/) (2022)
- [Uncovering Instabilities in Variational-Quantum Deep Q-Networks](https://mlsanity.com/paper/2202.05195/) (2022)
- [Multi-Agent Quantum Reinforcement Learning using Evolutionary Optimization](https://mlsanity.com/paper/2311.05546/) (2023)
- [Benchmarking Surrogate-Assisted Genetic Recommender Systems](https://mlsanity.com/paper/1908.02880/) (2019)
- [Towards Multi-Agent Reinforcement Learning using Quantum Boltzmann Machines](https://mlsanity.com/paper/2109.10900/) (2021)

## Abstract (arXiv)

In recent years, Multi-Agent Reinforcement Learning (MARL) has found application in numerous areas of science and industry, such as autonomous driving, telecommunications, and global health. Nevertheless, MARL suffers from, for instance, an exponential growth of dimensions. Inherent properties of quantum mechanics help to overcome these limitations, e.g., by significantly reducing the number of trainable parameters. Previous studies have developed an approach that uses gradient-free quantum Reinforcement Learning and evolutionary optimization for variational quantum circuits (VQCs) to reduce the trainable parameters and avoid barren plateaus as well as vanishing gradients. This leads to a significantly better performance of VQCs compared to classical neural networks with a similar number of trainable parameters and a reduction in the number of parameters by more than 97 % compared to similarly good neural networks. We extend an approach of K"olle et al. by proposing a Gate-Based, a Layer-Based, and a Prototype-Based concept to mutate and recombine VQCs. Our results show the best performance for mutation-only strategies and the Gate-Based approach. In particular, we observe a significantly better score, higher total and own collected coins, as well as a superior own coin rate for the best agent when evaluated in the Coin Game environment.