# ZO-JADE: Zeroth-order Curvature-Aware Multi-Agent Convex Optimization

arXiv: https://arxiv.org/abs/2303.07450 · HTML: https://mlsanity.com/paper/2303.07450/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Mar 2023.

## Built on by (most cited first, 2 of 2)

- [FedZeN: Towards superlinear zeroth-order federated learning via incremental Hessian estimation](https://mlsanity.com/paper/2309.17174/) (2023)
- [Network-GIANT: Fully distributed Newton-type optimization via harmonic Hessian consensus](https://mlsanity.com/paper/2305.07898/) (2023)

## Abstract (arXiv)

In this work we address the problem of convex optimization in a multi-agent setting where the objective is to minimize the mean of local cost functions whose derivatives are not available (e.g. black-box models). Moreover agents can only communicate with local neighbors according to a connected network topology. Zeroth-order (ZO) optimization has recently gained increasing attention in federated learning and multi-agent scenarios exploiting finite-difference approximations of the gradient using from 2 (directional gradient) to 2d (central difference full gradient) evaluations of the cost functions, where d is the dimension of the problem. The contribution of this work is to extend ZO distributed optimization by estimating the curvature of the local cost functions via finite-difference approximations. In particular, we propose a novel algorithm named ZO-JADE, that by adding just one extra point, i.e. 2d+1 in total, allows to simultaneously estimate the gradient and the diagonal of the local Hessian, which are then combined via average tracking consensus to obtain an approximated Jacobi descent. Guarantees of semi-global exponential stability are established via separation of time-scales. Extensive numerical experiments on real-world data confirm the efficiency and superiority of our algorithm with respect to several other distributed zeroth-order methods available in the literature based on only gradient estimates.