# A Study on Optimization Techniques for Variational Quantum Circuits in Reinforcement Learning

arXiv: https://arxiv.org/abs/2405.12354 · HTML: https://mlsanity.com/paper/2405.12354/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 28 references that are in the map.
- First posted on arXiv 20 May 2024.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 15 of 15)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [The effect of data encoding on the expressive power of variational quantum machine learning models](https://mlsanity.com/paper/2008.08605/) (2020)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [Parametrized quantum policies for reinforcement learning](https://mlsanity.com/paper/2103.05577/) (2021)
- [Layerwise learning for quantum neural networks](https://mlsanity.com/paper/2006.14904/) (2020)
- [Reinforcement Learning with Quantum Variational Circuits](https://mlsanity.com/paper/2008.07524/) (2020)
- [A Survey on Quantum Reinforcement Learning](https://mlsanity.com/paper/2211.03464/) (2022)
- [Variational Quantum Reinforcement Learning via Evolutionary Optimization](https://mlsanity.com/paper/2109.00540/) (2021)
- [Escaping from the Barren Plateau via Gaussian Initializations in Deep Variational Quantum Circuits](https://mlsanity.com/paper/2203.09376/) (2022)
- [Introduction to Quantum Reinforcement Learning: Theory and PennyLane-based Implementation](https://mlsanity.com/paper/2108.06849/) (2021)
- [Variational Quantum Soft Actor-Critic](https://mlsanity.com/paper/2112.11921/) (2021)
- [Policy Gradients using Variational Quantum Circuits](https://mlsanity.com/paper/2203.10591/) (2022)
- [Variational Quantum Soft Actor-Critic for Robotic Arm Control](https://mlsanity.com/paper/2212.11681/) (2022)
- [Improving Convergence for Quantum Variational Classifiers using Weight Re-Mapping](https://mlsanity.com/paper/2212.14807/) (2022)

## Built on by (most cited first, 8 of 8)

- [Dissecting Quantum Reinforcement Learning: A Systematic Evaluation of Key Components](https://mlsanity.com/paper/2511.17112/) (2025)
- [Investigating Parameter-Efficiency of Hybrid QuGANs Based on Geometric Properties of Generated Sea Route Graphs](https://mlsanity.com/paper/2501.08678/) (2025)
- [APGEM: Adaptive Policy-Guided Error Mitigation for Quantum Reinforcement Learning on a Real-World CVRP Case Study](https://mlsanity.com/paper/2609.18219/) (2026)
- [Optimization of Link Configuration for Satellite Communication Using Reinforcement Learning](https://mlsanity.com/paper/2501.08220/) (2025)
- [Supervised Quantum Machine Learning: A Future Outlook from Qubits to Enterprise Applications](https://mlsanity.com/paper/2505.24765/) (2025)
- [Illustration of Barren Plateaus in Quantum Computing](https://mlsanity.com/paper/2602.16558/) (2026)
- [Modeling quantum neural network gradient with reinforcement learning](https://mlsanity.com/paper/2609.31066/) (2026)
- [Context-Aware Error Mitigation Orchestration for Hybrid Quantum Reinforcement Learning on NISQ Systems](https://mlsanity.com/paper/2610.01253/) (2026)

## Abstract (arXiv)

Quantum Computing aims to streamline machine learning, making it more effective with fewer trainable parameters. This reduction of parameters can speed up the learning process and reduce the use of computational resources. However, in the current phase of quantum computing development, known as the noisy intermediate-scale quantum era (NISQ), learning is difficult due to a limited number of qubits and widespread quantum noise. To overcome these challenges, researchers are focusing on variational quantum circuits (VQCs). VQCs are hybrid algorithms that merge a quantum circuit, which can be adjusted through parameters, with traditional classical optimization techniques. These circuits require only few qubits for effective learning. Recent studies have presented new ways of applying VQCs to reinforcement learning, showing promising results that warrant further exploration. This study investigates the effects of various techniques -- data re-uploading, input scaling, output scaling -- and introduces exponential learning rate decay in the quantum proximal policy optimization algorithm's actor-VQC. We assess these methods in the popular Frozen Lake and Cart Pole environments. Our focus is on their ability to reduce the number of parameters in the VQC without losing effectiveness. Our findings indicate that data re-uploading and an exponential learning rate decay significantly enhance hyperparameter stability and overall performance. While input scaling does not improve parameter efficiency, output scaling effectively manages greediness, leading to increased learning speed and robustness.