mlsanityData to 14 Oct 2026

Online Algorithms and Policies Using Adaptive and Machine Learning Approaches

arXiv 2105.06577 ↗First posted 13 May 2021cs.LG math.DS math.OCcited by 3

Not on the map. Why?

Abstract (arXiv)

This paper considers the problem of real-time control and learning in dynamic systems subjected to parametric uncertainties. We propose a combination of a Reinforcement Learning (RL) based policy in the outer loop suitably chosen to ensure stability and optimality for the nominal dynamics, together with Adaptive Control (AC) in the inner loop so that in real-time AC contracts the closed-loop dynamics towards a stable trajectory traced out by RL. Two classes of nonlinear dynamic systems are considered, both of which are control-affine. The first class of dynamic systems utilizes equilibrium points %with expansion forms around these points and a Lyapunov approach while second class of nonlinear systems uses contraction theory. AC-RL controllers are proposed for both classes of systems and shown to lead to online policies that guarantee stability using a high-order tuner and accommodate parametric uncertainties and magnitude limits on the input. In addition to establishing a stability guarantee with real-time control, the AC-RL controller is also shown to lead to parameter learning with persistent excitation for the first class of systems. Numerical validations of all algorithms are carried out using a quadrotor landing task on a moving platform.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 24 of its 77 references that are in the map. First posted on arXiv 13 May 2021.

Builds on

24 of 77 references.

References · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  4. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 520
  5. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  6. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
  7. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  8. 2015A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insightscited by 206
  9. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 151
  10. 2016A Variational Perspective on Accelerated Methods in Optimizationcited by 110
  11. 2018Regret Bounds for Robust Adaptive Control of the Linear Quadratic Regulatorcited by 86
  12. 2019Deep Drone Racing: From Simulation to Reality with Domain Randomizationcited by 76
12 more
  1. 2017Towards Generalization and Simplicity in Continuous Controlcited by 68
  2. 2017Reinforcement Learning under Model Mismatchcited by 52
  3. 2015Control Contraction Metrics: Convex and Intrinsic Criteria for Nonlinear Feedback Designcited by 39
  4. 2021Adaptive-Control-Oriented Meta-Learning for Nonlinear Systemscited by 39
  5. 2018Direct Runge-Kutta Discretization Achieves Accelerationcited by 21
  6. 2020Reinforcement Learning with Fast Stabilization in Linear Dynamical Systemscited by 20
  7. 2020Convergence Proof for Actor-Critic Methods Applied to PPO and RUDDERcited by 19
  8. 2019From self-tuning regulators to reinforcement learning and back againcited by 17
  9. 2019Connections Between Adaptive Control and Optimization in Machine Learningcited by 7
  10. 2021Online Policies for Real-Time Control Using MRAC-RLcited by 7
  11. 2020A Stable High-order Tuner for General Convex Functionscited by 4
  12. 2020Accelerated Learning with Robustness to Adversarial Regressorscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2025Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approachescited by 5
  2. 2024Extremum-Seeking Action Selection for Accelerating Policy Optimizationcited by 1
  3. 2026Autopilot-Preserving Residual Q-Learning with HJB-Inspired Finite-Action Risk Filtering for Fixed-Wing UAV Command Supervisioncited by 0