# Dual Control for Approximate Bayesian Reinforcement Learning

arXiv: https://arxiv.org/abs/1510.03591 · HTML: https://mlsanity.com/paper/1510.03591/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2015.

## Built on by (most cited first, 9 of 9)

- [Active Uncertainty Reduction for Safe and Efficient Interaction Planning: A Shielding-Aware Dual Control Approach](https://mlsanity.com/paper/2302.00171/) (2023)
- [SHARP: Shielding-Aware Robust Planning for Safe and Efficient Human-Robot Interaction](https://mlsanity.com/paper/2110.00843/) (2021)
- [Active Uncertainty Reduction for Human-Robot Interaction: An Implicit Dual Control Approach](https://mlsanity.com/paper/2202.07720/) (2022)
- [Regret Analysis of Learning-Based MPC with Partially-Unknown Cost Function](https://mlsanity.com/paper/2108.02307/) (2021)
- [Conservative Dual Policy Optimization for Efficient Model-Based Reinforcement Learning](https://mlsanity.com/paper/2209.07676/) (2022)
- [Efficient Stochastic Optimal Control through Approximate Bayesian Input Inference](https://mlsanity.com/paper/2105.07693/) (2021)
- [Using Memory-Based Learning to Solve Tasks with State-Action Constraints](https://mlsanity.com/paper/2303.04327/) (2023)
- [Bayesian Learning for Dynamic Inference](https://mlsanity.com/paper/2301.00032/) (2022)
- [Generalized Information Gathering Under Dynamics Uncertainty](https://mlsanity.com/paper/2601.21988/) (2026)

## Abstract (arXiv)

Control of non-episodic, finite-horizon dynamical systems with uncertain dynamics poses a tough and elementary case of the exploration-exploitation trade-off. Bayesian reinforcement learning, reasoning about the effect of actions and future observations, offers a principled solution, but is intractable. We review, then extend an old approximate approach from control theory---where the problem is known as dual control---in the context of modern regression methods, specifically generalized linear regression. Experiments on simulated systems show that this framework offers a useful approximation to the intractable aspects of Bayesian RL, producing structured exploration strategies that differ from standard RL approaches. We provide simple examples for the use of this framework in (approximate) Gaussian process regression and feedforward neural networks for the control of exploration.