Optimisation of Structured Neural Controller Based on Continuous-Time Policy Gradient
Not on the map. Why?
Abstract (arXiv)
This study presents a policy optimisation framework for structured nonlinear control of continuous-time (deterministic) dynamic systems. The proposed approach prescribes a structure for the controller based on relevant scientific knowledge (such as Lyapunov stability theory or domain experiences) while considering the tunable elements inside the given structure as the point of parametrisation with neural networks. To optimise a cost represented as a function of the neural network weights, the proposed approach utilises the continuous-time policy gradient method based on adjoint sensitivity analysis as a means for correct and performant computation of cost gradient. This enables combining the stability, robustness, and physical interpretability of an analytically-derived structure for the feedback controller with the representational flexibility and optimised resulting performance provided by machine learning techniques. Such a hybrid paradigm for fixed-structure control synthesis is particularly useful for optimising adaptive nonlinear controllers to achieve improved performance in online operation, an area where the existing theory prevails the design of structure while lacking clear analytical understandings about tuning of the gains and the uncertainty model basis functions that govern the performance characteristics. Numerical experiments on aerospace applications illustrate the utility of the structured nonlinear controller optimisation framework.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 19 of its 37 references that are in the map. First posted on arXiv 17 Jan 2022.
Builds on
19 of 37 references.
References · 19
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
- 2020Universal Differential Equations for Scientific Machine Learningcited by 345
- 2019A Differentiable Programming System to Bridge Machine Learning and Scientific Computingcited by 81
- 2019Pontryagin Differentiable Programming: An End-to-End Learning and Control Frameworkcited by 48
- 2018Don't Unroll Adjoint: Differentiating SSA-Form Programscited by 44
- 2021Continuous-Time Model-Based Reinforcement Learningcited by 39
- 2019An Online Learning Approach to Model Predictive Controlcited by 29
- 2020Hamilton-Jacobi Deep Q-Learning for Deterministic Continuous-Time Systems with Lipschitz Continuous Controlscited by 29
- 2020Stein Variational Model Predictive Controlcited by 29
- 2016Real-time optimal control via Deep Neural Networks: study on landing problemscited by 27
7 more
- 2021Safe Pontryagin Differentiable Programmingcited by 25
- 2021Youla-REN: Learning Nonlinear Feedback Policies with Robust Stability Guaranteescited by 10
- 2020Adaptive Meta-Learning for Identification of Rover-Terrain Dynamicscited by 8
- 2020Faster Policy Learning with Continuous-Time Gradientscited by 8
- 2019Hamilton-Jacobi-Bellman Equations for Q-Learning in Continuous Timecited by 6
- 2020Hamilton-Jacobi-Bellman Equations for Maximum Entropy Optimal Controlcited by 3
- 2019A Domain-Knowledge-Aided Deep Reinforcement Learning Approach for Flight Control Designcited by 2
Built on by
Citing papers per year: 2022: 1.
1 paper.