mlsanityData to 7 Oct 2026

Minibatch Stochastic Three Points Method for Unconstrained Smooth Minimization

arXiv 2209.07883 ↗First posted 16 Sep 2022math.OC cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a new zero order optimization method called minibatch stochastic three points (MiSTP) method to solve an unconstrained minimization problem in a setting where only an approximation of the objective function evaluation is possible. It is based on the recently proposed stochastic three points (STP) method (Bergou et al., 2020). At each iteration, MiSTP generates a random search direction in a similar manner to STP, but chooses the next iterate based solely on the approximation of the objective function rather than its exact evaluations. We also analyze our method's complexity in the nonconvex and convex cases and evaluate its performance on multiple machine learning tasks.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 17 of its 31 references that are in the map. First posted on arXiv 16 Sep 2022.

Builds on

17 of 31 references.

References · 17

  1. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  2. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
  3. 2019SGD: General Analysis and Improved Ratescited by 206
  4. 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 146
  5. 2016Stochastic Variance Reduction for Nonconvex Optimizationcited by 143
  6. 2018Zeroth-Order Stochastic Variance Reduction for Nonconvex Optimizationcited by 109
  7. 2018Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systemscited by 56
  8. 2019Distributed Reinforcement Learning for Decentralized Linear Quadratic Control: A Derivative-Free Policy Optimization Approachcited by 33
  9. 2019Stochastic Three Points Method for Unconstrained Smooth Minimizationcited by 24
  10. 2018Autotune: A Derivative-free Optimization Framework for Hyperparameter Tuningcited by 20
  11. 2013Convergence of trust-region methods based on probabilistic modelscited by 10
  12. 2019A Stochastic Derivative-Free Optimization Method with Importance Sampling: Theory and Learning to Controlcited by 9
5 more
  1. 2019A Stochastic Derivative Free Optimization Method with Momentumcited by 9
  2. 2020An Empirical Study of Derivative-Free-Optimization Algorithms for Targeted Black-Box Attacks in Deep Neural Networkscited by 8
  3. 2018A stochastic Levenberg-Marquardt method using random models with complexity resultscited by 5
  4. 2017Black-Box Optimization in Machine Learning with Trust Region Based Derivative Free Algorithmcited by 3
  5. 2016Convergence Rate Analysis of a Stochastic Trust Region Method via Submartingalescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 0; 2025: 1; 2026: 3.

6 papers.

  1. 2023A Guide Through the Zoo of Biased SGDcited by 39
  2. 2023Towards a Better Theoretical Understanding of Independent Subnetwork Trainingcited by 6
  3. 2025Stochastic Optimization with Random Searchcited by 0
  4. 2026Riemannian Dueling Optimizationcited by 0
  5. 2026Trust the Direction, Search the Step: Zero-and-First-Order Methods for LLM Fine-Tuningcited by 0
  6. 2026TACO: Ternary Absolute-max Column-wise One-sparse Optimizer for LLM Fine-Tuningcited by 0