# A Computationally Efficient Limited Memory CMA-ES for Large Scale Optimization

arXiv: https://arxiv.org/abs/1404.5520 · HTML: https://mlsanity.com/paper/1404.5520/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2014.

## Built on by (most cited first, 5 of 5)

- [DesignX: Human-Competitive Algorithm Designer for Black-Box Optimization](https://mlsanity.com/paper/2505.17866/) (2025)
- [Solving Deep Reinforcement Learning Tasks with Evolution Strategies and Linear Policy Networks](https://mlsanity.com/paper/2402.06912/) (2024)
- [On the Level Sets and Invariance of Neural Tuning Landscapes](https://mlsanity.com/paper/2212.13285/) (2022)
- [Reinforcement learning in densely recurrent biological networks](https://mlsanity.com/paper/2508.09618/) (2025)
- [Inference-Time Optimization of Prompt Embeddings in Diffusion Models: A Comparison of sep-CMA-ES and Adam](https://mlsanity.com/paper/2511.03913/) (2025)

## Abstract (arXiv)

We propose a computationally efficient limited memory Covariance Matrix Adaptation Evolution Strategy for large scale optimization, which we call the LM-CMA-ES. The LM-CMA-ES is a stochastic, derivative-free algorithm for numerical optimization of non-linear, non-convex optimization problems in continuous domain. Inspired by the limited memory BFGS method of Liu and Nocedal (1989), the LM-CMA-ES samples candidate solutions according to a covariance matrix reproduced from m direction vectors selected during the optimization process. The decomposition of the covariance matrix into Cholesky factors allows to reduce the time and memory complexity of the sampling to O(mn), where n is the number of decision variables. When n is large (e.g., n > 1000), even relatively small values of m (e.g., m=20,30) are sufficient to efficiently solve fully non-separable problems and to reduce the overall run-time.