# Adaptive State-Dependent Diffusion for Derivative-Free Optimization

arXiv: https://arxiv.org/abs/2302.04370 · HTML: https://mlsanity.com/paper/2302.04370/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 41 references that are in the map.
- First posted on arXiv 8 Feb 2023.

## Builds on (most cited first, 5 of 5)

- [A consensus-based global optimization method for high dimensional machine learning problems](https://mlsanity.com/paper/1909.09249/) (2019)
- [Consensus-Based Optimization Methods Converge Globally](https://mlsanity.com/paper/2103.15130/) (2021)
- [Scalable Subspace Methods for Derivative-Free Nonlinear Least-Squares Optimization](https://mlsanity.com/paper/2102.12016/) (2021)
- [Trends in Consensus-based optimization](https://mlsanity.com/paper/2104.01383/) (2021)
- [An Algebraically Converging Stochastic Gradient Descent Algorithm for Global Optimization](https://mlsanity.com/paper/2204.05923/) (2022)

## Built on by (most cited first, 5 of 5)

- [Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints](https://mlsanity.com/paper/2402.18012/) (2024)
- [Gradient is All You Need? How Consensus-Based Optimization can be Interpreted as a Stochastic Relaxation of Gradient Descent](https://mlsanity.com/paper/2306.09778/) (2023)
- [Sampling with Adaptive Variance for Multimodal Distributions](https://mlsanity.com/paper/2411.15220/) (2024)
- [Accelerating Convergence in Global Non-Convex Optimization with Reversible Diffusion](https://mlsanity.com/paper/2305.11493/) (2023)
- [Gradient-free Riemannian Langevin Sampler](https://mlsanity.com/paper/2607.07519/) (2026)

## Abstract (arXiv)

This paper develops and analyzes a stochastic derivative-free optimization strategy. A key feature is the state-dependent adaptive variance. We prove global convergence in probability with algebraic rate and give the quantitative results in numerical examples. A striking fact is that convergence is achieved without explicit information of the gradient and even without comparing different objective function values as in established methods such as the simplex method and simulated annealing. It can otherwise be compared to annealing with state-dependent temperature.