# Limited-Memory Matrix Adaptation for Large Scale Black-box Optimization

arXiv: https://arxiv.org/abs/1705.06693 · HTML: https://mlsanity.com/paper/1705.06693/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2017.

## Built on by (most cited first, 4 of 4)

- [evosax: JAX-based Evolution Strategies](https://mlsanity.com/paper/2212.04180/) (2022)
- [Generating Master Faces for Dictionary Attacks with a Network-Assisted Latent Space Evolution](https://mlsanity.com/paper/2108.01077/) (2021)
- [Generating 2D and 3D Master Faces for Dictionary Attacks with a Network-Assisted Latent Space Evolution](https://mlsanity.com/paper/2211.13964/) (2022)
- [Learning Instance-Specific Parameters of Black-Box Models Using Differentiable Surrogates](https://mlsanity.com/paper/2407.17530/) (2024)

## Abstract (arXiv)

The Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is a popular method to deal with nonconvex and/or stochastic optimization problems when the gradient information is not available. Being based on the CMA-ES, the recently proposed Matrix Adaptation Evolution Strategy (MA-ES) provides a rather surprising result that the covariance matrix and all associated operations (e.g., potentially unstable eigendecomposition) can be replaced in the CMA-ES by a updated transformation matrix without any loss of performance. In order to further simplify MA-ES and reduce its Obig(n2big) time and storage complexity to Obig(nlog(n)big), we present the Limited-Memory Matrix Adaptation Evolution Strategy (LM-MA-ES) for efficient zeroth order large-scale optimization. The algorithm demonstrates state-of-the-art performance on a set of established large-scale benchmarks. We explore the algorithm on the problem of generating adversarial inputs for a (non-smooth) random forest classifier, demonstrating a surprising vulnerability of the classifier.