# Leveraging Memory Effects and Gradient Information in Consensus-Based Optimization: On Global Convergence in Mean-Field Law

arXiv: https://arxiv.org/abs/2211.12184 · HTML: https://mlsanity.com/paper/2211.12184/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2022.

## Built on by (most cited first, 5 of 5)

- [FedCBO: Reaching Group Consensus in Clustered Federated Learning through Consensus-based Optimization](https://mlsanity.com/paper/2305.02894/) (2023)
- [Quantifying Concentration Phenomena of Mean-Field Transformers in the Low-Temperature Regime](https://mlsanity.com/paper/2605.10931/) (2026)
- [Gradient is All You Need? How Consensus-Based Optimization can be Interpreted as a Stochastic Relaxation of Gradient Descent](https://mlsanity.com/paper/2306.09778/) (2023)
- [Consensus-based optimization for closed-box adversarial attacks and a connection to evolution strategies](https://mlsanity.com/paper/2506.24048/) (2025)
- [Defending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimization](https://mlsanity.com/paper/2412.02535/) (2024)

## Abstract (arXiv)

In this paper we study consensus-based optimization (CBO), a versatile, flexible and customizable optimization method suitable for performing nonconvex and nonsmooth global optimizations in high dimensions. CBO is a multi-particle metaheuristic, which is effective in various applications and at the same time amenable to theoretical analysis thanks to its minimalistic design. The underlying dynamics, however, is flexible enough to incorporate different mechanisms widely used in evolutionary computation and machine learning, as we show by analyzing a variant of CBO which makes use of memory effects and gradient information. We rigorously prove that this dynamics converges to a global minimizer of the objective function in mean-field law for a vast class of functions under minimal assumptions on the initialization of the method. The proof in particular reveals how to leverage further, in some applications advantageous, forces in the dynamics without loosing provable global convergence. To demonstrate the benefit of the herein investigated memory effects and gradient information in certain applications, we present numerical evidence for the superiority of this CBO variant in applications such as machine learning and compressed sensing, which en passant widen the scope of applications of CBO.