mlsanityData to 14 Oct 2026

Equipping Black-Box Policies with Model-Based Advice for Stable Nonlinear Control

arXiv 2206.01341 ↗First posted 2 Jun 2022cs.LG cs.SY eess.SYcited by 6

Not on the map. Why?

Abstract (arXiv)

Machine-learned black-box policies are ubiquitous for nonlinear control problems. Meanwhile, crude model information is often available for these problems from, e.g., linear approximations of nonlinear dynamics. We study the problem of equipping a black-box control policy with model-based advice for nonlinear control on a single trajectory. We first show a general negative result that a naive convex combination of a black-box policy and a linear model-based policy can lead to instability, even if the two policies are both stabilizing. We then propose an adaptive λ-confident policy, with a coefficient λ indicating the confidence in a black-box policy, and prove its stability. With bounded nonlinearity, in addition, we show that the adaptive λ-confident policy achieves a bounded competitive ratio when a black-box policy is near-optimal. Finally, we propose an online learning approach to implement the adaptive λ-confident policy and verify its efficacy in case studies about the CartPole problem and a real-world electric vehicle (EV) charging problem with data bias due to COVID-19.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 24 of its 38 references that are in the map. First posted on arXiv 2 Jun 2022.

Builds on

24 of 38 references.

References · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2015Trust Region Policy Optimizationcited by 2,580
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  6. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  7. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  8. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  9. 2018Competitive caching with machine learned advicecited by 195
  10. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 167
  11. 2017On the Sample Complexity of the Linear Quadratic Regulatorcited by 159
  12. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 156
12 more
  1. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 151
  2. 2018Temporal Difference Models: Model-Free Deep RL for Model-Based Controlcited by 89
  3. 2019Safe Reinforcement Learning Using Robust MPCcited by 85
  4. 2020Optimal Robustness-Consistency Trade-offs for Learning-Augmented Online Algorithmscited by 64
  5. 2016Learning Model Predictive Control for iterative tasks. A Data-Driven Control Frameworkcited by 39
  6. 2019Control Regularization for Reduced Variance Reinforcement Learningcited by 37
  7. 2019Unsupervised Curricula for Visual Meta-Reinforcement Learningcited by 34
  8. 2019Model-Based Reinforcement Learning with Adversarial Training for Online Recommendationcited by 33
  9. 2018Stability-certified reinforcement learning: A control-theoretic perspectivecited by 31
  10. 2021Robustness and Consistency in Linear Quadratic Control with Untrusted Predictionscited by 14
  11. 2020Competitive Control with Delayed Imperfect Informationcited by 8
  12. 2019On the Equivalence of Youla, System-level and Input-output Parameterizationscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 1; 2026: 1.

6 papers.

  1. 2023Physics-Regulated Deep Reinforcement Learning: Invariant Embeddingscited by 10
  2. 2023Robust Learning for Smoothed Online Convex Optimization with Feedback Delaycited by 3
  3. 2024Simplex-enabled Safe Continual Learning Machinecited by 2
  4. 2024Physics-model-guided Worst-case Sampling for Safe Reinforcement Learningcited by 0
  5. 2025Real-DRL: Teach and Learn in Realitycited by 0
  6. 2026Cognitively-Grounded On-Device Runtime Learning for Ground Robots in Unknown Physical Environmentscited by 0