# Synthesizing Neural Network Controllers with Probabilistic Model based Reinforcement Learning

arXiv: https://arxiv.org/abs/1803.02291 · HTML: https://mlsanity.com/paper/1803.02291/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Mar 2018.

## Built on by (most cited first, 9 of 9)

- [QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMs](https://mlsanity.com/paper/2510.11696/) (2025)
- [The Surprising Difficulty of Search in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2601.21306/) (2026)
- [Low Emission Building Control with Zero-Shot Reinforcement Learning](https://mlsanity.com/paper/2208.06385/) (2022)
- [Decomposing Control Lyapunov Functions for Efficient Reinforcement Learning](https://mlsanity.com/paper/2403.12210/) (2024)
- [GrASP: Gradient-Based Affordance Selection for Planning](https://mlsanity.com/paper/2202.04772/) (2022)
- [Uncertainty Aware Learning from Demonstrations in Multiple Contexts using Bayesian Neural Networks](https://mlsanity.com/paper/1903.05697/) (2019)
- [Video2Skill: Adapting Events in Demonstration Videos to Skills in an Environment using Cyclic MDP Homomorphisms](https://mlsanity.com/paper/2109.03813/) (2021)
- [High Performance Across Two Atari Paddle Games Using the Same Perceptual Control Architecture Without Training](https://mlsanity.com/paper/2108.01895/) (2021)
- [Low Emission Building Control with Zero-Shot Reinforcement Learning](https://mlsanity.com/paper/2206.14191/) (2022)

## Abstract (arXiv)

We present an algorithm for rapidly learning controllers for robotics systems. The algorithm follows the model-based reinforcement learning paradigm, and improves upon existing algorithms; namely Probabilistic learning in Control (PILCO) and a sample-based version of PILCO with neural network dynamics (Deep-PILCO). We propose training a neural network dynamics model using variational dropout with truncated Log-Normal noise. This allows us to obtain a dynamics model with calibrated uncertainty, which can be used to simulate controller executions via rollouts. We also describe set of techniques, inspired by viewing PILCO as a recurrent neural network model, that are crucial to improve the convergence of the method. We test our method on a variety of benchmark tasks, demonstrating data-efficiency that is competitive with PILCO, while being able to optimize complex neural network controllers. Finally, we assess the performance of the algorithm for learning motor controllers for a six legged autonomous underwater vehicle. This demonstrates the potential of the algorithm for scaling up the dimensionality and dataset sizes, in more complex control tasks.