# Algorithmic Optimality Guarantees for Nonsmooth H_∞ Output-Feedback Policy Search

arXiv: https://arxiv.org/abs/2609.06197 · HTML: https://mlsanity.com/paper/2609.06197/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 26 references that are in the map.
- First posted on arXiv 5 Sep 2026.

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [Policy Optimization for H2 Linear Control with H_∞ Robustness Guarantee: Implicit Regularization and Global Convergence](https://mlsanity.com/paper/1910.09496/) (2019)
- [A gradient sampling method with complexity guarantees for Lipschitz functions in high and low dimensions](https://mlsanity.com/paper/2112.06969/) (2021)
- [Globally Convergent Policy Search over Dynamic Filters for Output Estimation](https://mlsanity.com/paper/2202.11659/) (2022)
- [Global Convergence of Direct Policy Search for State-Feedback H_∞ Robust Control: A Revisit of Nonsmooth Synthesis with Goldstein Subdifferential](https://mlsanity.com/paper/2210.11577/) (2022)
- [Connectivity of the Feasible and Sublevel Sets of Dynamic Output Feedback Control with Robustness Constraints](https://mlsanity.com/paper/2203.11177/) (2022)
- [Benign Nonconvex Landscapes in Optimal and Robust Control, Part I: Global Optimality](https://mlsanity.com/paper/2312.15332/) (2023)
- [Output-feedback Synthesis Orbit Geometry: Quotient Manifolds and LQG Direct Policy Optimization](https://mlsanity.com/paper/2403.17157/) (2024)
- [Policy Optimization in Robust Control: Weak Convexity and Subgradient Methods](https://mlsanity.com/paper/2509.25633/) (2025)
- [Lipschitz minimization and the Goldstein modulus](https://mlsanity.com/paper/2405.12655/) (2024)

## Abstract (arXiv)

We study continuous-time full-order dynamic output-feedback H_∞ policy search, a nonconvex and nonsmooth problem. Direct policy search is a central paradigm in reinforcement learning and continuous control, but rigorous guarantees remain scarce in robust output-feedback settings. The H_∞ problem is a canonical benchmark because it captures disturbance attenuation and robustness while exposing the hard nonsmooth geometry of policy-space optimization. We prove that on the exact identity-gauge slice of the extended convex lift, varepsilon-stationarity yields O(varepsilon)-suboptimality on compact exact slices, which in turn yields convergence-rate guarantees for nonsmooth policy-search methods. This result addresses the finite-time optimality-gap question raised by Guo and Hu [2022] in the more general dynamic output-feedback H_∞ policy-search setting. We further use the established value equivalence supplied by extended convex lifting to formulate a nonstrict-feasibility bisection method with one final strict-feasibility recovery step, yielding an explicit varepsilon-optimal stabilizing controller. These results provide a quantitative and algorithmic strengthening of prior qualitative optimality theory for nonsmooth H_∞ policy search.