# Continuous Control with Contexts, Provably

arXiv: https://arxiv.org/abs/1910.13614 · HTML: https://mlsanity.com/paper/1910.13614/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2019.

## Built on by (most cited first, 3 of 3)

- [Joint Learning-Based Stabilization of Multiple Unknown Linear Systems](https://mlsanity.com/paper/2201.01387/) (2022)
- [Provably Efficient Lifelong Reinforcement Learning with Linear Function Approximation](https://mlsanity.com/paper/2206.00270/) (2022)
- [Scaling Distributed Multi-task Reinforcement Learning with Experience Sharing](https://mlsanity.com/paper/2307.05834/) (2023)

## Abstract (arXiv)

A fundamental challenge in artificial intelligence is to build an agent that generalizes and adapts to unseen environments. A common strategy is to build a decoder that takes the context of the unseen new environment as input and generates a policy accordingly. The current paper studies how to build a decoder for the fundamental continuous control task, linear quadratic regulator (LQR), which can model a wide range of real-world physical environments. We present a simple algorithm for this problem, which uses upper confidence bound (UCB) to refine the estimate of the decoder and balance the exploration-exploitation trade-off. Theoretically, our algorithm enjoys a widetildeOleft(√Tright) regret bound in the online setting where T is the number of environments the agent played. This also implies after playing widetildeOleft(1/ε2right) environments, the agent is able to transfer the learned knowledge to obtain an ε-suboptimal policy for an unseen environment. To our knowledge, this is first provably efficient algorithm to build a decoder in the continuous control setting. While our main focus is theoretical, we also present experiments that demonstrate the effectiveness of our algorithm.