# System stabilization with policy optimization on unstable latent manifolds

arXiv: https://arxiv.org/abs/2407.06418 · HTML: https://mlsanity.com/paper/2407.06418/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 26 of its 77 references that are in the map.
- First posted on arXiv 8 Jul 2024.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 26 of 26)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [Survey of multifidelity methods in uncertainty propagation, inference, and optimization](https://mlsanity.com/paper/1806.10761/) (2018)
- [Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems](https://mlsanity.com/paper/1912.08177/) (2019)
- [Multiscale Simulations of Complex Systems by Learning their Effective Dynamics](https://mlsanity.com/paper/2006.13431/) (2020)
- [A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder](https://mlsanity.com/paper/2009.11990/) (2020)
- [Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systems](https://mlsanity.com/paper/1812.08305/) (2018)
- [Extracting Sparse High-Dimensional Dynamics from Limited Data](https://mlsanity.com/paper/1707.08528/) (2017)
- [Convergence and sample complexity of gradient methods for the model-free linear quadratic regulator problem](https://mlsanity.com/paper/1912.11899/) (2019)
- [Distributed Reinforcement Learning for Decentralized Linear Quadratic Control: A Derivative-Free Policy Optimization Approach](https://mlsanity.com/paper/1912.09135/) (2019)
- [A framework for data-driven solution and parameter estimation of PDEs using conditional generative adversarial networks](https://mlsanity.com/paper/2105.13136/) (2021)
- [Quadratic Approximation Manifold for Mitigating the Kolmogorov Barrier in Nonlinear Projection-Based Model Order Reduction](https://mlsanity.com/paper/2204.02462/) (2022)
- [Non-intrusive reduced order modeling of natural convection in porous media using convolutional autoencoders: comparison with linear subspace techniques](https://mlsanity.com/paper/2107.11460/) (2021)
- [Sampling low-dimensional Markovian dynamics for pre-asymptotically recovering reduced models from data with operator inference](https://mlsanity.com/paper/1908.11233/) (2019)
- [Physics-informed regularization and structure preservation for learning stable reduced models from data with operator inference](https://mlsanity.com/paper/2107.02597/) (2021)
- [Nonlinear Model Order Reduction via Lifting Transformations and Proper Orthogonal Decomposition](https://mlsanity.com/paper/1808.02086/) (2018)
- [CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations](https://mlsanity.com/paper/2402.14646/) (2024)
- [Data-driven Structured Realization](https://mlsanity.com/paper/1611.02072/) (2016)
- [On the sample complexity of stabilizing linear dynamical systems from data](https://mlsanity.com/paper/2203.00474/) (2022)
- [Model-free control](https://mlsanity.com/paper/1305.7085/) (2013)
- [Learning low-dimensional dynamical-system models from noisy frequency-response data with Loewner rational interpolation](https://mlsanity.com/paper/1910.00110/) (2019)
- [Robust output-feedback stabilization for incompressible flows using low-dimensional H∞-controllers](https://mlsanity.com/paper/2103.01608/) (2021)
- [Context-aware controller inference for stabilizing dynamical systems from scarce data](https://mlsanity.com/paper/2207.11049/) (2022)
- [Multi-fidelity robust controller design with gradient sampling](https://mlsanity.com/paper/2205.15050/) (2022)

## Built on by (most cited first, 2 of 2)

- [Learning Stabilizing Policies via an Unstable Subspace Representation](https://mlsanity.com/paper/2505.01348/) (2025)
- [An adaptive data sampling strategy for stabilizing dynamical systems via controller inference](https://mlsanity.com/paper/2506.01816/) (2025)

## Abstract (arXiv)

Stability is a basic requirement when studying the behavior of dynamical systems. However, stabilizing dynamical systems via reinforcement learning is challenging because only little data can be collected over short time horizons before instabilities are triggered and data become meaningless. This work introduces a reinforcement learning approach that is formulated over latent manifolds of unstable dynamics so that stabilizing policies can be trained from few data samples. The unstable manifolds are minimal in the sense that they contain the lowest dimensional dynamics that are necessary for learning policies that guarantee stabilization. This is in stark contrast to generic latent manifolds that aim to approximate all -- stable and unstable -- system dynamics and thus are higher dimensional and often require higher amounts of data. Experiments demonstrate that the proposed approach stabilizes even complex physical systems from few data samples for which other methods that operate either directly in the system state space or on generic latent manifolds fail.