# Combining Model-Based and Model-Free Methods for Nonlinear Control: A Provably Convergent Policy Gradient Approach

arXiv: https://arxiv.org/abs/2006.07476 · HTML: https://mlsanity.com/paper/2006.07476/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2020.

## Built on by (most cited first, 5 of 5)

- [Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges](https://mlsanity.com/paper/2102.01168/) (2021)
- [Stabilizing Dynamical Systems via Policy Gradient Methods](https://mlsanity.com/paper/2110.06418/) (2021)
- [Policy Optimization for PDE Control with a Warm Start](https://mlsanity.com/paper/2403.01005/) (2024)
- [Imitation Game: A Model-based and Imitation Learning Deep Reinforcement Learning Hybrid](https://mlsanity.com/paper/2404.01794/) (2024)
- [Reinforcement Twinning for Hybrid Control of Flapping-Wing Drones](https://mlsanity.com/paper/2505.18201/) (2025)

## Abstract (arXiv)

Model-free learning-based control methods have seen great success recently. However, such methods typically suffer from poor sample complexity and limited convergence guarantees. This is in sharp contrast to classical model-based control, which has a rich theory but typically requires strong modeling assumptions. In this paper, we combine the two approaches to achieve the best of both worlds. We consider a dynamical system with both linear and non-linear components and develop a novel approach to use the linear model to define a warm start for a model-free, policy gradient method. We show this hybrid approach outperforms the model-based controller while avoiding the convergence issues associated with model-free approaches via both numerical experiments and theoretical analyses, in which we derive sufficient conditions on the non-linear component such that our approach is guaranteed to converge to the (nearly) global optimal controller.