# Fitting a Linear Control Policy to Demonstrations with a Kalman Constraint

arXiv: https://arxiv.org/abs/2001.07572 · HTML: https://mlsanity.com/paper/2001.07572/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jan 2020.

## Built on by (most cited first, 9 of 9)

- [Learning Model Predictive Controllers with Real-Time Attention for Real-World Navigation](https://mlsanity.com/paper/2209.10780/) (2022)
- [CCIL: Context-conditioned imitation learning for urban driving](https://mlsanity.com/paper/2305.02649/) (2023)
- [On Imitation Learning of Linear Control Policies: Enforcing Stability and Robustness Constraints via LMI Conditions](https://mlsanity.com/paper/2103.12945/) (2021)
- [Towards a Theoretical Foundation of Policy Optimization for Learning Control Policies](https://mlsanity.com/paper/2210.04810/) (2022)
- [An Improved Data Augmentation Scheme for Model Predictive Control Policy Approximation](https://mlsanity.com/paper/2303.05607/) (2023)
- [Imitation Learning of Stabilizing Policies for Nonlinear Systems](https://mlsanity.com/paper/2109.10854/) (2021)
- [Dissipative Imitation Learning for Robust Dynamic Output Feedback](https://mlsanity.com/paper/2210.00979/) (2022)
- [Learning to control from expert demonstrations](https://mlsanity.com/paper/2203.05012/) (2022)
- [Dissipative Imitation Learning for Discrete Dynamic Output Feedback Control with Sparse Data Sets](https://mlsanity.com/paper/2309.06658/) (2023)

## Abstract (arXiv)

We consider the problem of learning a linear control policy for a linear dynamical system, from demonstrations of an expert regulating the system. The standard approach to this problem is policy fitting, which fits a linear policy by minimizing a loss function between the demonstrations and the policy's outputs plus a regularization function that encodes prior knowledge. Despite its simplicity, this method fails to learn policies with low or even finite cost when there are few demonstrations. We propose to add an additional constraint to policy fitting, that the policy is the solution to some LQR problem, i.e., optimal in the stochastic control sense for some choice of quadratic cost. We refer to this constraint as a Kalman constraint. Policy fitting with a Kalman constraint requires solving an optimization problem with convex cost and bilinear constraints. We propose a heuristic method, based on the alternating direction method of multipliers (ADMM), to approximately solve this problem. Numerical experiments demonstrate that adding the Kalman constraint allows us to learn good, i.e., low cost, policies even when very few data are available.