# Temporally smoothed incremental model-based heuristic dynamic programming for command-filtered cascaded online learning flight control

arXiv: https://arxiv.org/abs/2507.04346 · HTML: https://mlsanity.com/paper/2507.04346/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 53 references that are in the map.
- First posted on arXiv 6 Jul 2025.

## Builds on (most cited first, 11 of 11)

- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Projection-Based Constrained Policy Optimization](https://mlsanity.com/paper/2010.03152/) (2020)
- [Safe Policies for Reinforcement Learning via Primal-Dual Methods](https://mlsanity.com/paper/1911.09101/) (2019)
- [Regularizing Action Policies for Smooth Control with Reinforcement Learning](https://mlsanity.com/paper/2012.06644/) (2020)
- [Achieving Zero Constraint Violation for Constrained Reinforcement Learning via Primal-Dual Approach](https://mlsanity.com/paper/2109.06332/) (2021)
- [L2C2: Locally Lipschitz Continuous Constraint towards Stable and Smooth Reinforcement Learning](https://mlsanity.com/paper/2202.07152/) (2022)
- [A Primal-Dual-Critic Algorithm for Offline Constrained Reinforcement Learning](https://mlsanity.com/paper/2306.07818/) (2023)
- [Soft Actor-Critic Deep Reinforcement Learning for Fault Tolerant Flight Control](https://mlsanity.com/paper/2202.09262/) (2022)
- [Equivariant Reinforcement Learning Frameworks for Quadrotor Low-Level Control](https://mlsanity.com/paper/2502.20500/) (2025)
- [Complete stability analysis of a heuristic ADP control design](https://mlsanity.com/paper/1308.3282/) (2013)
- [Deep deterministic policy gradient with symmetric data augmentation for lateral attitude tracking control of a fixed-wing aircraft](https://mlsanity.com/paper/2407.11077/) (2024)

## Abstract (arXiv)

Approximate Dynamic Programming (ADP) enables online adaptation of control laws through adaptive critic methods. However, its application to online learning Angle-of-Attack (AoA) tracking control remains challenging due to oscillatory control actions under nonlinear dynamics, which may induce system oscillations, degrade tracking performance, and increase actuator effort. To address this challenge, this paper proposes two innovations for online learning flight control: (1) incorporating temporal-scale policy smoothness regularization into the Incremental Model-based Heuristic Dynamic Programming (IHDP) framework; and (2) employing a low-pass filter to attenuate high-frequency pitch-rate commands. Furthermore, a primal-dual approach is developed to adaptively adjust the smoothness penalty weight in the policy objective according to a prescribed smoothness criterion. Tracking control simulations demonstrate that the proposed methods reduce control system oscillations, improve tracking performance, and exhibit post-training policy robustness to model uncertainties.