# Relative Entropy Regularized Reinforcement Learning for Efficient Encrypted Policy Synthesis

arXiv: https://arxiv.org/abs/2506.12358 · HTML: https://mlsanity.com/paper/2506.12358/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 27 references that are in the map.
- First posted on arXiv 14 Jun 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 9 of 9)

- [Cloud-based Quadratic Optimization with Partially Homomorphic Encryption](https://mlsanity.com/paper/1809.02267/) (2018)
- [SARSA(0) Reinforcement Learning over Fully Homomorphic Encryption](https://mlsanity.com/paper/2002.00506/) (2020)
- [Encrypted Value Iteration and Temporal Difference Learning over Leveled Homomorphic Encryption](https://mlsanity.com/paper/2103.11065/) (2021)
- [Cloud-based MPC with Encrypted Data](https://mlsanity.com/paper/1803.09891/) (2018)
- [Dynamic controller that operates over homomorphically encrypted data for infinite time horizon](https://mlsanity.com/paper/1912.07362/) (2019)
- [Input-Output History Feedback Controller for Encrypted Control with Leveled Fully Homomorphic Encryption](https://mlsanity.com/paper/2109.10718/) (2021)
- [Encrypted dynamic control with unlimited operating time via FIR filters](https://mlsanity.com/paper/2112.02931/) (2021)
- [Encrypted extremum seeking for privacy-preserving PID tuning as-a-Service](https://mlsanity.com/paper/2207.04442/) (2022)
- [Ring-LWE based encrypted controller with unlimited number of recursive multiplications and effect of error growth](https://mlsanity.com/paper/2406.14372/) (2024)

## Abstract (arXiv)

We propose an efficient encrypted policy synthesis to develop privacy-preserving model-based reinforcement learning. We first demonstrate that the relative-entropy-regularized reinforcement learning framework offers a computationally convenient linear and ``min-free'' structure for value iteration, enabling a direct and efficient integration of fully homomorphic encryption with bootstrapping into policy synthesis. Convergence and error bounds are analyzed as encrypted policy synthesis propagates errors under the presence of encryption-induced errors including quantization and bootstrapping. Theoretical analysis is validated by numerical simulations. Results demonstrate the effectiveness of the RERL framework in integrating FHE for encrypted policy synthesis.