# Combining Benefits from Trajectory Optimization and Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1910.09667 · HTML: https://mlsanity.com/paper/1910.09667/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2019.

## Built on by (most cited first, 2 of 2)

- [Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians](https://mlsanity.com/paper/2102.13073/) (2021)
- [Optimizing Trajectories for Highway Driving with Offline Reinforcement Learning](https://mlsanity.com/paper/2203.10949/) (2022)

## Abstract (arXiv)

Recent breakthroughs both in reinforcement learning and trajectory optimization have made significant advances towards real world robotic system deployment. Reinforcement learning (RL) can be applied to many problems without needing any modeling or intuition about the system, at the cost of high sample complexity and the inability to prove any metrics about the learned policies. Trajectory optimization (TO) on the other hand allows for stability and robustness analyses on generated motions and trajectories, but is only as good as the often over-simplified derived model, and may have prohibitively expensive computation times for real-time control. This paper seeks to combine the benefits from these two areas while mitigating their drawbacks by (1) decreasing RL sample complexity by using existing knowledge of the problem with optimal control, and (2) providing an upper bound estimate on the time-to-arrival of the combined learned-optimized policy, allowing online policy deployment at any point in the training process by using the TO as a worst-case scenario action. This method is evaluated for a car model, with applicability to any mobile robotic system. A video showing policy execution comparisons can be found at https://youtu.be/mv2xw83NyWU .