# Boosting Trust Region Policy Optimization by Normalizing Flows Policy

arXiv: https://arxiv.org/abs/1809.10326 · HTML: https://mlsanity.com/paper/1809.10326/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2018.

## Built on by (most cited first, 11 of 11)

- [GenPO: Generative Diffusion Models Meet On-Policy Reinforcement Learning](https://mlsanity.com/paper/2505.18763/) (2025)
- [S2AC: Energy-Based Reinforcement Learning with Stein Soft Actor Critic](https://mlsanity.com/paper/2405.00987/) (2024)
- [Learning Sampling Distributions for Model Predictive Control](https://mlsanity.com/paper/2212.02587/) (2022)
- [CEIP: Combining Explicit and Implicit Priors for Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/2210.09496/) (2022)
- [Soft Actor-Critic With Integer Actions](https://mlsanity.com/paper/2109.08512/) (2021)
- [GenPO++: Generative Policy Optimization with Jacobian-free Likelihood Ratios](https://mlsanity.com/paper/2606.06967/) (2026)
- [Improving Actor-Critic Reinforcement Learning via Hamiltonian Monte Carlo Method](https://mlsanity.com/paper/2103.12020/) (2021)
- [Bellman: A Toolbox for Model-Based Reinforcement Learning in TensorFlow](https://mlsanity.com/paper/2103.14407/) (2021)
- [EnTRPO: Trust Region Policy Optimization Method with Entropy Regularization](https://mlsanity.com/paper/2110.13373/) (2021)
- [Policy Gradient With Serial Markov Chain Reasoning](https://mlsanity.com/paper/2210.06766/) (2022)
- [Entropy Augmented Reinforcement Learning](https://mlsanity.com/paper/2208.09322/) (2022)

## Abstract (arXiv)

We propose to improve trust region policy search with normalizing flows policy. We illustrate that when the trust region is constructed by KL divergence constraints, normalizing flows policy generates samples far from the 'center' of the previous policy iterate, which potentially enables better exploration and helps avoid bad local optima. Through extensive comparisons, we show that the normalizing flows policy significantly improves upon baseline architectures especially on high-dimensional tasks with complex dynamics.