# Variable Gain Gradient Descent-based Reinforcement Learning for Robust Optimal Tracking Control of Uncertain Nonlinear System with Input-Constraints

arXiv: https://arxiv.org/abs/1911.04157 · HTML: https://mlsanity.com/paper/1911.04157/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2019.

## Built on by (most cited first, 1 of 1)

- [Super Level Sets and Exponential Decay: A Synergistic Approach to Stable Neural Network Training](https://mlsanity.com/paper/2409.16769/) (2024)

## Abstract (arXiv)

In recent times, a variety of Reinforcement Learning (RL) algorithms have been proposed for optimal tracking problem of continuous time nonlinear systems with input constraints. Most of these algorithms are based on the notion of uniform ultimate boundedness (UUB) stability, in which normally higher learning rates are avoided in order to restrict oscillations in state error to smaller values. However, this comes at the cost of higher convergence time of critic neural network weights. This paper addresses that problem by proposing a novel tuning law containing a variable gain gradient descent for critic neural network that can adjust the learning rate based on Hamilton-Jacobi-Bellman (HJB) error. By allowing high learning rate the proposed variable gain gradient descent tuning law could improve the convergence time of critic neural network weights. Simultaneously, it also results in tighter residual set, on which trajectories of augmented system converge to, leading to smaller oscillations in state error. A tighter bound for UUB stability of the proposed update mechanism is proved. Numerical studies are then presented to validate the robust Reinforcement Learning control scheme in controlling a continuous time nonlinear system.