# Zeroth-order Optimization for Composite Problems with Functional Constraints

arXiv: https://arxiv.org/abs/2112.11420 · HTML: https://mlsanity.com/paper/2112.11420/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2021.

## Built on by (most cited first, 4 of 4)

- [Stochastic Inexact Augmented Lagrangian Method for Nonconvex Expectation Constrained Optimization](https://mlsanity.com/paper/2212.09513/) (2022)
- [Oracle Complexity of Single-Loop Switching Subgradient Methods for Non-Smooth Weakly Convex Functional Constrained Optimization](https://mlsanity.com/paper/2301.13314/) (2023)
- [An Empirical Evaluation of Zeroth-Order Optimization Methods on AI-driven Molecule Optimization](https://mlsanity.com/paper/2210.16099/) (2022)
- [How to Boost Any Loss Function](https://mlsanity.com/paper/2407.02279/) (2024)

## Abstract (arXiv)

In many real-world problems, first-order (FO) derivative evaluations are too expensive or even inaccessible. For solving these problems, zeroth-order (ZO) methods that only need function evaluations are often more efficient than FO methods or sometimes the only options. In this paper, we propose a novel zeroth-order inexact augmented Lagrangian method (ZO-iALM) to solve black-box optimization problems, which involve a composite (i.e., smooth+nonsmooth) objective and functional constraints. Under a certain regularity condition (also assumed by several existing works on FO methods), the query complexity of our ZO-iALM is tildeO(dvarepsilon-3) to find an varepsilon-KKT point for problems with a nonconvex objective and nonconvex constraints, and tildeO(dvarepsilon-2.5) for nonconvex problems with convex constraints, where d is the variable dimension. This appears to be the first work that develops an iALM-based ZO method for functional constrained optimization and meanwhile achieves query complexity results matching the best-known FO complexity results up to a factor of d. With an extensive experimental study, we show the effectiveness of our method. The applications of our method span from classical optimization problems to practical machine learning examples such as resource allocation in sensor networks and adversarial example generation.