# Imitation Learning from Nonlinear MPC via the Exact Q-Loss and its Gauss-Newton Approximation

arXiv: https://arxiv.org/abs/2304.01782 · HTML: https://mlsanity.com/paper/2304.01782/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 10 of its 27 references that are in the map.
- First posted on arXiv 3 Apr 2023.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [acados: a modular open-source framework for fast embedded optimal control](https://mlsanity.com/paper/1910.13753/) (2019)
- [Learning an Approximate Model Predictive Controller with Guarantees](https://mlsanity.com/paper/1806.04167/) (2018)
- [MPC-Net: A First Principles Guided Policy Search](https://mlsanity.com/paper/1909.05197/) (2019)
- [Imitation Learning from MPC for Quadrupedal Multi-Gait Control](https://mlsanity.com/paper/2103.14331/) (2021)
- [Deep Learning Explicit Differentiable Predictive Control Laws for Buildings](https://mlsanity.com/paper/2107.11843/) (2021)
- [Stability Verification of Neural Network Controllers using Mixed-Integer Programming](https://mlsanity.com/paper/2206.13374/) (2022)
- [End-to-End Imitation Learning with Safety Guarantees using Control Barrier Functions](https://mlsanity.com/paper/2212.11365/) (2022)
- [Deep Learning-Based Model Predictive Control for Resonant Power Converters](https://mlsanity.com/paper/1810.04872/) (2018)

## Built on by (most cited first, 5 of 5)

- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [AC4MPC: Actor-Critic Reinforcement Learning for Nonlinear Model Predictive Control](https://mlsanity.com/paper/2406.03995/) (2024)
- [Differentiable Nonlinear Model Predictive Control](https://mlsanity.com/paper/2505.01353/) (2025)
- [CACTO-BIC: Scalable Actor-Critic Learning via Biased Sampling and GPU-Accelerated Trajectory Optimization](https://mlsanity.com/paper/2602.19699/) (2026)
- [Merging model-based control with multi-agent reinforcement learning for multi-agent cooperative teaming strategies](https://mlsanity.com/paper/2606.06011/) (2026)

## Abstract (arXiv)

This work presents a novel loss function for learning nonlinear Model Predictive Control policies via Imitation Learning. Standard approaches to Imitation Learning neglect information about the expert and generally adopt a loss function based on the distance between expert and learned controls. In this work, we present a loss based on the Q-function directly embedding the performance objectives and constraint satisfaction of the associated Optimal Control Problem (OCP). However, training a Neural Network with the Q-loss requires solving the associated OCP for each new sample. To alleviate the computational burden, we derive a second Q-loss based on the Gauss-Newton approximation of the OCP resulting in a faster training time. We validate our losses against Behavioral Cloning, the standard approach to Imitation Learning, on the control of a nonlinear system with constraints. The final results show that the Q-function-based losses significantly reduce the amount of constraint violations while achieving comparable or better closed-loop costs.