# Unit Tests for Stochastic Optimization

arXiv: https://arxiv.org/abs/1312.6055 · HTML: https://mlsanity.com/paper/1312.6055/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Dec 2013.

## Built on by (most cited first, 6 of 6)

- [The Risks of Machine Learning Systems](https://mlsanity.com/paper/2204.09852/) (2022)
- [NeuroEvoBench: Benchmarking Evolutionary Optimizers for Deep Learning Applications](https://mlsanity.com/paper/2311.02394/) (2023)
- [Novel ANN method for solving ordinary and fractional Black-Scholes equation](https://mlsanity.com/paper/2105.11240/) (2021)
- [Automatically Differentiable Random Coefficient Logistic Demand Estimation](https://mlsanity.com/paper/2106.04636/) (2021)
- [Deep Learning-based Segmentation of Cerebral Aneurysms in 3D TOF-MRA using Coarse-to-Fine Framework](https://mlsanity.com/paper/2110.13432/) (2021)
- [Empowering the trustworthiness of ML-based critical systems through engineering activities](https://mlsanity.com/paper/2209.15438/) (2022)

## Abstract (arXiv)

Optimization by stochastic gradient descent is an important component of many large-scale machine learning algorithms. A wide variety of such optimization algorithms have been devised; however, it is unclear whether these algorithms are robust and widely applicable across many different optimization landscapes. In this paper we develop a collection of unit tests for stochastic optimization. Each unit test rapidly evaluates an optimization algorithm on a small-scale, isolated, and well-understood difficulty, rather than in real-world scenarios where many such issues are entangled. Passing these unit tests is not sufficient, but absolutely necessary for any algorithms with claims to generality or robustness. We give initial quantitative and qualitative results on numerous established algorithms. The testing framework is open-source, extensible, and easy to apply to new algorithms.