# Model-Free μ-Synthesis: A Nonsmooth Optimization Perspective

arXiv: https://arxiv.org/abs/2402.11654 · HTML: https://mlsanity.com/paper/2402.11654/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 55 references that are in the map.
- First posted on arXiv 18 Feb 2024.

## Builds on (most cited first, 24 of 24)

- [Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming](https://mlsanity.com/paper/1309.5549/) (2013)
- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systems](https://mlsanity.com/paper/1812.08305/) (2018)
- [Policy Gradient Methods for the Noisy Linear Quadratic Regulator over a Finite Horizon](https://mlsanity.com/paper/2011.10300/) (2020)
- [Policy Optimization for H2 Linear Control with H_∞ Robustness Guarantee: Implicit Regularization and Global Convergence](https://mlsanity.com/paper/1910.09496/) (2019)
- [Convergence and sample complexity of gradient methods for the model-free linear quadratic regulator problem](https://mlsanity.com/paper/1912.11899/) (2019)
- [Gradient-Free Methods for Deterministic and Stochastic Nonsmooth Nonconvex Optimization](https://mlsanity.com/paper/2209.05045/) (2022)
- [LQR through the Lens of First Order Methods: Discrete-time Case](https://mlsanity.com/paper/1907.08921/) (2019)
- [Analysis of the Optimization Landscape of Linear Quadratic Gaussian (LQG) Control](https://mlsanity.com/paper/2102.04393/) (2021)
- [Optimizing Static Linear Feedback: Gradient Method](https://mlsanity.com/paper/2004.09875/) (2020)
- [Stabilizing Dynamical Systems via Policy Gradient Methods](https://mlsanity.com/paper/2110.06418/) (2021)
- [Gradient Sampling Methods for Nonsmooth Optimization](https://mlsanity.com/paper/1804.11003/) (2018)
- [Learning the Globally Optimal Distributed LQ Regulator](https://mlsanity.com/paper/1912.08774/) (2019)
- [Derivative-Free Policy Optimization for Linear Risk-Sensitive and Robust Control Design: Implicit Regularization and Sample Complexity](https://mlsanity.com/paper/2101.01041/) (2021)
- [Optimization Landscape of Gradient Descent for Discrete-time Static Output Feedback](https://mlsanity.com/paper/2109.13132/) (2021)
- [Primal-dual Learning for the Model-free Risk-constrained Linear Quadratic Regulator](https://mlsanity.com/paper/2011.10931/) (2020)
- [Convergence Guarantees of Policy Optimization Methods for Markovian Jump Linear Systems](https://mlsanity.com/paper/2002.04090/) (2020)
- [Global Convergence of Direct Policy Search for State-Feedback H_∞ Robust Control: A Revisit of Nonsmooth Synthesis with Goldstein Subdifferential](https://mlsanity.com/paper/2210.11577/) (2022)
- [Policy Learning of MDPs with Mixed Continuous/Discrete Variables: A Case Study on Model-Free Control of Markovian Jump Systems](https://mlsanity.com/paper/2006.03116/) (2020)
- [Connectivity of the Feasible and Sublevel Sets of Dynamic Output Feedback Control with Robustness Constraints](https://mlsanity.com/paper/2203.11177/) (2022)
- [Model-Free μ Synthesis via Adversarial Reinforcement Learning](https://mlsanity.com/paper/2111.15537/) (2021)
- [Escaping High-order Saddles in Policy Optimization for Linear Quadratic Gaussian (LQG) Control](https://mlsanity.com/paper/2204.00912/) (2022)
- [Global Convergence Using Policy Gradient Methods for Model-free Markovian Jump Linear Quadratic Control](https://mlsanity.com/paper/2111.15228/) (2021)
- [Benign Nonconvex Landscapes in Optimal and Robust Control, Part I: Global Optimality](https://mlsanity.com/paper/2312.15332/) (2023)

## Abstract (arXiv)

In this paper, we revisit model-free policy search on an important robust control benchmark, namely μ-synthesis. In the general output-feedback setting, there do not exist convex formulations for this problem, and hence global optimality guarantees are not expected. Apkarian (2011) presented a nonconvex nonsmooth policy optimization approach for this problem, and achieved state-of-the-art design results via using subgradient-based policy search algorithms which generate update directions in a model-based manner. Despite the lack of convexity and global optimality guarantees, these subgradient-based policy search methods have led to impressive numerical results in practice. Built upon such a policy optimization persepctive, our paper extends these subgradient-based search methods to a model-free setting. Specifically, we examine the effectiveness of two model-free policy optimization strategies: the model-free non-derivative sampling method and the zeroth-order policy search with uniform smoothing. We performed an extensive numerical study to demonstrate that both methods consistently replicate the design outcomes achieved by their model-based counterparts. Additionally, we provide some theoretical justifications showing that convergence guarantees to stationary points can be established for our model-free μ-synthesis under some assumptions related to the coerciveness of the cost function. Overall, our results demonstrate that derivative-free policy optimization offers a competitive and viable approach for solving general output-feedback μ-synthesis problems in the model-free setting.