System stabilization with policy optimization on unstable latent manifolds
Not on the map. Why?
Abstract (arXiv)
Stability is a basic requirement when studying the behavior of dynamical systems. However, stabilizing dynamical systems via reinforcement learning is challenging because only little data can be collected over short time horizons before instabilities are triggered and data become meaningless. This work introduces a reinforcement learning approach that is formulated over latent manifolds of unstable dynamics so that stabilizing policies can be trained from few data samples. The unstable manifolds are minimal in the sense that they contain the lowest dimensional dynamics that are necessary for learning policies that guarantee stabilization. This is in stark contrast to generic latent manifolds that aim to approximate all -- stable and unstable -- system dynamics and thus are higher dimensional and often require higher amounts of data. Experiments demonstrate that the proposed approach stabilizes even complex physical systems from few data samples for which other methods that operate either directly in the system state space or on generic latent manifolds fail.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 26 of its 77 references that are in the map. First posted on arXiv 8 Jul 2024.
Builds on
26 of 77 references.
References · 26
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
- 2018Survey of multifidelity methods in uncertainty propagation, inference, and optimizationcited by 141
- 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 129
- 2019Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systemscited by 96
- 2020Multiscale Simulations of Complex Systems by Learning their Effective Dynamicscited by 86
- 2020A fast and accurate physics-informed neural network reduced order model with shallow masked autoencodercited by 80
- 2018Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systemscited by 56
- 2017Extracting Sparse High-Dimensional Dynamics from Limited Datacited by 50
- 2019Convergence and sample complexity of gradient methods for the model-free linear quadratic regulator problemcited by 35
- 2019Distributed Reinforcement Learning for Decentralized Linear Quadratic Control: A Derivative-Free Policy Optimization Approachcited by 33
14 more
- 2021A framework for data-driven solution and parameter estimation of PDEs using conditional generative adversarial networkscited by 30
- 2022Quadratic Approximation Manifold for Mitigating the Kolmogorov Barrier in Nonlinear Projection-Based Model Order Reductioncited by 29
- 2021Non-intrusive reduced order modeling of natural convection in porous media using convolutional autoencoders: comparison with linear subspace techniquescited by 23
- 2019Sampling low-dimensional Markovian dynamics for pre-asymptotically recovering reduced models from data with operator inferencecited by 20
- 2021Physics-informed regularization and structure preservation for learning stable reduced models from data with operator inferencecited by 16
- 2018Nonlinear Model Order Reduction via Lifting Transformations and Proper Orthogonal Decompositioncited by 14
- 2024CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equationscited by 12
- 2016Data-driven Structured Realizationcited by 11
- 2022On the sample complexity of stabilizing linear dynamical systems from datacited by 8
- 2013Model-free controlcited by 6
- 2019Learning low-dimensional dynamical-system models from noisy frequency-response data with Loewner rational interpolationcited by 6
- 2021Robust output-feedback stabilization for incompressible flows using low-dimensional H∞-controllerscited by 3
- 2022Context-aware controller inference for stabilizing dynamical systems from scarce datacited by 3
- 2022Multi-fidelity robust controller design with gradient samplingcited by 1
Built on by
Citing papers per year: 2025: 2.
2 papers.