# Neural Co-state Projection Regulator: A Model-free Paradigm for Real-time Optimal Control with Input Constraints

arXiv: https://arxiv.org/abs/2508.00283 · HTML: https://mlsanity.com/paper/2508.00283/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 7 of its 25 references that are in the map.
- First posted on arXiv 1 Aug 2025.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 7 of 7)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Quantifying Generalization in Reinforcement Learning](https://mlsanity.com/paper/1812.02341/) (2018)
- [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019)
- [Resource Constrained Deep Reinforcement Learning](https://mlsanity.com/paper/1812.00600/) (2018)
- [A Machine Learning Enhanced Algorithm for the Optimal Landing Problem](https://mlsanity.com/paper/2203.06753/) (2022)
- [Neural Co-state Regulator: A Data-Driven Paradigm for Real-time Optimal Control with Input Constraints](https://mlsanity.com/paper/2507.12259/) (2025)

## Abstract (arXiv)

Learning-based approaches, notably Reinforcement Learning (RL), have shown promise for solving optimal control tasks without explicit system models. However, these approaches are often sample-inefficient, sensitive to reward design and hyperparameters, and prone to poor generalization, especially under input constraints. To address these challenges, we introduce the neural co-state projection regulator (NCPR), a model-free learning-based optimal control framework that is grounded in Pontryagin's Minimum Principle (PMP) and capable of solving quadratic regulator problems in nonlinear control-affine systems with input constraints. In this framework, a neural network (NN) is trained in a self-supervised setting to take the current state of the system as input and predict a finite-horizon trajectory of projected co-states (i.e., the co-state weighted by the system's input gain). Subsequently, only the first element of the NN's prediction is extracted to solve a lightweight quadratic program (QP). This workflow is executed in a feedback control setting, allowing real-time computation of control actions that satisfy both input constraints and first-order optimality conditions. We test the proposed learning-based model-free quadratic regulator on (1) a unicycle model robot reference tracking problem and (2) a pendulum swing-up task. For comparison, reinforcement learning is used on both tasks; and for context, a model-based controller is used in the unicycle model example. Our method demonstrates superior generalizability in terms of both unseen system states and varying input constraints, and also shows improved sampling efficiency.