# H_∞ Model-free Reinforcement Learning with Robust Stability Guarantee

arXiv: https://arxiv.org/abs/1911.02875 · HTML: https://mlsanity.com/paper/1911.02875/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2019.

## Built on by (most cited first, 7 of 7)

- [Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimization](https://mlsanity.com/paper/2105.08881/) (2021)
- [Recurrent Neural Network Controllers Synthesis with Stability Guarantees for Partially Observed Systems](https://mlsanity.com/paper/2109.03861/) (2021)
- [On Imitation Learning of Linear Control Policies: Enforcing Stability and Robustness Constraints via LMI Conditions](https://mlsanity.com/paper/2103.12945/) (2021)
- [FI-ODE: Certifiably Robust Forward Invariance in Neural ODEs](https://mlsanity.com/paper/2210.16940/) (2022)
- [Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learning](https://mlsanity.com/paper/2203.12048/) (2022)
- [Revisiting PGD Attacks for Stability Analysis of Large-Scale Nonlinear Systems and Perception-Based Control](https://mlsanity.com/paper/2201.00801/) (2022)
- [Zero-Sum Positional Differential Games as a Framework for Robust Reinforcement Learning: Deep Q-Learning Approach](https://mlsanity.com/paper/2405.02044/) (2024)

## Abstract (arXiv)

Reinforcement learning is showing great potentials in robotics applications, including autonomous driving, robot manipulation and locomotion. However, with complex uncertainties in the real-world environment, it is difficult to guarantee the successful generalization and sim-to-real transfer of learned policies theoretically. In this paper, we introduce and extend the idea of robust stability and H_∞ control to design policies with both stability and robustness guarantee. Specifically, a sample-based approach for analyzing the Lyapunov stability and performance robustness of a learning-based control system is proposed. Based on the theoretical results, a maximum entropy algorithm is developed for searching Lyapunov function and designing a policy with provable robust stability guarantee. Without any specific domain knowledge, our method can find a policy that is robust to various uncertainties and generalizes well to different test environments. In our experiments, we show that our method achieves better robustness to both large impulsive disturbances and parametric variations in the environment than the state-of-art results in both robust and generic RL, as well as classic control. Anonymous code is available to reproduce the experimental results at https://github.com/RobustStabilityGuaranteeRL/RobustStabilityGuaranteeRL.