# Efficient Implementation of Reinforcement Learning over Homomorphic Encryption

arXiv: https://arxiv.org/abs/2504.09335 · HTML: https://mlsanity.com/paper/2504.09335/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 2 of its 19 references that are in the map.
- First posted on arXiv 12 Apr 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 2 of 2)

- [SARSA(0) Reinforcement Learning over Fully Homomorphic Encryption](https://mlsanity.com/paper/2002.00506/) (2020)
- [Encrypted Value Iteration and Temporal Difference Learning over Leveled Homomorphic Encryption](https://mlsanity.com/paper/2103.11065/) (2021)

## Abstract (arXiv)

We investigate encrypted control policy synthesis over the cloud. While encrypted control implementations have been studied previously, we focus on the less explored paradigm of privacy-preserving control synthesis, which can involve heavier computations ideal for cloud outsourcing. We classify control policy synthesis into model-based, simulator-driven, and data-driven approaches and examine their implementation over fully homomorphic encryption (FHE) for privacy enhancements. A key challenge arises from comparison operations (min or max) in standard reinforcement learning algorithms, which are difficult to execute over encrypted data. This observation motivates our focus on Relative-Entropy-regularized reinforcement learning (RL) problems, which simplifies encrypted evaluation of synthesis algorithms due to their comparison-free structures. We demonstrate how linearly solvable value iteration, path integral control, and Z-learning can be readily implemented over FHE. We conduct a case study of our approach through numerical simulations of encrypted Z-learning in a grid world environment using the CKKS encryption scheme, showing convergence with acceptable approximation error. Our work suggests the potential for secure and efficient cloud-based reinforcement learning.