# Derivative-Free Optimization via Adaptive Sampling Strategies

arXiv: https://arxiv.org/abs/2404.11893 · HTML: https://mlsanity.com/paper/2404.11893/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Apr 2024.

## Built on by (most cited first, 6 of 6)

- [Enhancing Efficiency of Safe Reinforcement Learning via Sample Manipulation](https://mlsanity.com/paper/2405.20860/) (2024)
- [Fast Unconstrained Optimization via Hessian Averaging and Adaptive Gradient Sampling Methods](https://mlsanity.com/paper/2408.07268/) (2024)
- [Derivative-Free Optimization via Finite Difference Approximation: An Experimental Study](https://mlsanity.com/paper/2411.00112/) (2024)
- [Learning Dynamics of Zeroth-Order Optimization: A Kernel Perspective](https://mlsanity.com/paper/2605.03373/) (2026)
- [Enhanced Derivative-Free Optimization Using Adaptive Correlation-Induced Finite Difference Estimators](https://mlsanity.com/paper/2502.20819/) (2025)
- [ZoAQ: Adaptive Zeroth-Order Querying via Query-Reuse Coupling](https://mlsanity.com/paper/2609.22115/) (2026)

## Abstract (arXiv)

In this paper, we present a novel derivative-free optimization framework for solving unconstrained stochastic optimization problems. Many problems in fields ranging from simulation optimization to reinforcement learning involve settings where only stochastic function values are obtained via an oracle with no available gradient information, necessitating the usage of derivative-free optimization methodologies. Our approach includes estimating gradients using stochastic function evaluations and integrating adaptive sampling techniques to control the accuracy in these stochastic approximations. We consider various gradient estimation techniques including standard finite difference, Gaussian smoothing, sphere smoothing, randomized coordinate finite difference, and randomized subspace finite difference methods. We provide theoretical convergence guarantees for our framework and analyze the worst-case iteration and sample complexities associated with each gradient estimation method. Finally, we demonstrate the empirical performance of the methods on logistic regression and nonlinear least squares problems.