mlsanityData to 7 Oct 2026

Stabilizing Dynamical Systems via Policy Gradient Methods

arXiv 2110.06418 ↗First posted 13 Oct 2021eess.SY cs.LG cs.SYcited by 24

Not on the map. Why?

Abstract (arXiv)

Stabilizing an unknown control system is one of the most fundamental problems in control systems engineering. In this paper, we provide a simple, model-free algorithm for stabilizing fully observed dynamical systems. While model-free methods have become increasingly popular in practice due to their simplicity and flexibility, stabilization via direct policy search has received surprisingly little attention. Our algorithm proceeds by solving a series of discounted LQR problems, where the discount factor is gradually increased. We prove that this method efficiently recovers a stabilizing controller for linear systems, and for smooth, nonlinear systems within a neighborhood of their equilibria. Our approach overcomes a significant limitation of prior work, namely the need for a pre-given stabilizing control policy. We empirically evaluate the effectiveness of our approach on common control benchmarks.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 22 of its 45 references that are in the map. First posted on arXiv 13 Oct 2021.

Builds on

22 of 45 references.

References · 22

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2018Learning Dexterous In-Hand Manipulationcited by 643
  4. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  5. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 211
  6. 2017On the Sample Complexity of the Linear Quadratic Regulatorcited by 151
  7. 2020Naive Exploration is Optimal for Online LQRcited by 111
  8. 2020Information Theoretic Regret Bounds for Online Nonlinear Controlcited by 87
  9. 2018Regret Bounds for Robust Adaptive Control of the Linear Quadratic Regulatorcited by 86
  10. 2020Improper Learning for Non-Stochastic Controlcited by 83
  11. 2019Learning Linear-Quadratic Regulators Efficiently with only √T Regretcited by 67
  12. 2018Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systemscited by 56
10 more
  1. 2019The Nonstochastic Control Problemcited by 52
  2. 2020Black-Box Control for Linear Dynamical Systemscited by 43
  3. 2018The Gap Between Model-Based and Model-Free Methods on the Linear Quadratic Regulator: An Asymptotic Viewpointcited by 36
  4. 2019Convergence and sample complexity of gradient methods for the model-free linear quadratic regulator problemcited by 35
  5. 2021Derivative-Free Policy Optimization for Linear Risk-Sensitive and Robust Control Design: Implicit Regularization and Sample Complexitycited by 15
  6. 2018Finite Time Adaptive Stabilization of LQ Systemscited by 7
  7. 2019Randomized Algorithms for Data-Driven Stabilization of Stochastic Linear Systemscited by 6
  8. 2020Combining Model-Based and Model-Free Methods for Nonlinear Control: A Provably Convergent Policy Gradient Approachcited by 5
  9. 2021Towards a Dimension-Free Understanding of Adaptive Linear Controlcited by 2
  10. 2019Escaping Locally Optimal Decentralized Control Polices via Dampingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 9; 2024: 3; 2025: 5; 2026: 1.

24 papers.

Sort:

  1. 2022Statistical Learning Theory for Control: A Finite Sample Perspectivecited by 52
  2. 2023Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregressioncited by 14
  3. 2024Meta-Learning Linear Quadratic Regulators: A Policy Gradient MAML Approach for Model-free LQRcited by 14
  4. 2023Learning the Kalman Filter with Fine-Grained Sample Complexitycited by 9
  5. 2023Revisiting LQR Control from the Perspective of Receding-Horizon Policy Gradientcited by 9
  6. 2022A Complete Characterization of Linear Estimators for Offline Policy Evaluationcited by 7
  7. 2022How are policy gradient methods affected by the limits of control?cited by 6
  8. 2022Towards a Theoretical Foundation of Policy Optimization for Learning Control Policiescited by 6
  9. 2022Global Convergence of Direct Policy Search for State-Feedback H_∞ Robust Control: A Revisit of Nonsmooth Synthesis with Goldstein Subdifferentialcited by 6
  10. 2023Global Convergence of Receding-Horizon Policy Search in Learning Estimator Designscited by 4
  11. 2023Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithmscited by 4
  12. 2022Context-aware controller inference for stabilizing dynamical systems from scarce datacited by 3
12 more
  1. 2025Learning Stabilizing Policies via an Unstable Subspace Representationcited by 3
  2. 2023Stabilizing reinforcement learning control: A modular framework for optimizing over all stable behaviorcited by 2
  3. 2024Learning to Stabilize Unknown LTI Systems on a Single Trajectory under Stochastic Noisecited by 2
  4. 2023Oracle Complexity Reduction for Model-free LQR: A Stochastic Variance-Reduced Policy Gradient Approachcited by 1
  5. 2023On the Hardness of Learning to Stabilize Linear Systemscited by 1
  6. 2023Accelerated Optimization Landscape of Linear-Quadratic Regulatorcited by 0
  7. 2024Model-Free μ-Synthesis: A Nonsmooth Optimization Perspectivecited by 0
  8. 2025Sample Complexity of Linear Quadratic Regulator Without Initial Stabilitycited by 0
  9. 2025Policy Gradient for LQR with Domain Randomizationcited by 0
  10. 2025An adaptive data sampling strategy for stabilizing dynamical systems via controller inferencecited by 0
  11. 2025Adversarially Robust Multitask Adaptive Controlcited by 0
  12. 2026Model-Free Output Feedback Stabilization via Policy Gradient Methodscited by 0