# Scalable Derivative-Free Optimization Algorithms with Low-Dimensional Subspace Techniques

arXiv: https://arxiv.org/abs/2501.04536 · HTML: https://mlsanity.com/paper/2501.04536/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jan 2025.

## Built on by (most cited first, 1 of 1)

- [Robust and Efficient Zeroth-Order LLM Fine-Tuning via Adaptive Bayesian Subspace Optimizer](https://mlsanity.com/paper/2601.01452/) (2026)

## Abstract (arXiv)

We re-introduce a derivative-free subspace optimization framework originating from Chapter 5 of the Ph.D. thesis [Z. Zhang, On Derivative-Free Optimization Methods, Ph.D. thesis, Chinese Academy of Sciences, Beijing, 2012] of the author under the supervision of Ya-xiang Yuan. At each iteration, the framework defines a (low-dimensional) subspace based on an approximate gradient, and then solves a subproblem in this subspace to generate a new iterate. We sketch the global convergence and worst-case complexity analysis of the framework, elaborate on its implementation, and present some numerical results on solving problems with dimensions as high as 10^4 using only inaccurate function values.