# BOND: License to Train with Black-Box Functions

arXiv: https://arxiv.org/abs/2510.25074 · HTML: https://mlsanity.com/paper/2510.25074/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 56 references that are in the map.
- First posted on arXiv 29 Oct 2025.

## Builds on (most cited first, 18 of 18)

- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://mlsanity.com/paper/1708.03999/) (2017)
- [Black-box Adversarial Attacks with Limited Queries and Information](https://mlsanity.com/paper/1804.08598/) (2018)
- [Optimal rates for zero-order convex optimization: the power of two function evaluations](https://mlsanity.com/paper/1312.2139/) (2013)
- [A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning](https://mlsanity.com/paper/2006.06224/) (2020)
- [Next Generation Reservoir Computing](https://mlsanity.com/paper/2106.07688/) (2021)
- [ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimization](https://mlsanity.com/paper/1910.06513/) (2019)
- [Stochastic Zeroth-order Optimization in High Dimensions](https://mlsanity.com/paper/1710.10551/) (2017)
- [Equilibrated adaptive learning rates for non-convex optimization](https://mlsanity.com/paper/1502.04390/) (2015)
- [Strength of Minibatch Noise in SGD](https://mlsanity.com/paper/2102.05375/) (2021)
- [A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning](https://mlsanity.com/paper/2307.15092/) (2023)
- [Analogue and Physical Reservoir Computing Using Water Waves](https://mlsanity.com/paper/2306.09095/) (2023)
- [A New One-Point Residual-Feedback Oracle For Black-Box Learning and Control](https://mlsanity.com/paper/2006.10820/) (2020)
- [Multiplexed gradient descent: Fast online training of modern datasets on hardware neural networks without backpropagation](https://mlsanity.com/paper/2303.03986/) (2023)
- [Stochastic Zeroth-order Optimization via Variance Reduction method](https://mlsanity.com/paper/1805.11811/) (2018)
- [Scaling Recurrent Neural Networks to a Billion Parameters with Zero-Order Optimization](https://mlsanity.com/paper/2505.17852/) (2025)
- [Convergence Rates of Finite Difference Stochastic Approximation Algorithms](https://mlsanity.com/paper/1506.09211/) (2015)
- [Perturbative Gradient Training: A novel training paradigm for bridging the gap between deep neural networks and physical reservoir computing](https://mlsanity.com/paper/2506.04523/) (2025)

## Abstract (arXiv)

We introduce Bounded Numerical Differentiation (BOND), a perturbative method for estimating the gradients of black-box functions. BOND is distinguished by its formulation, which adaptively bounds perturbations to ensure accurate sign estimation, and by its implementation, which operates at black-box interfaces. This enables BOND to be more accurate and scalable compared to existing methods, facilitating end-to-end training of architectures that incorporate non-autodifferentiable modules. We observe that these modules, implemented in our experiments as frozen networks, can enhance model performance without increasing the number of trainable parameters. Our findings highlight the potential of leveraging fixed transformations to expand model capacity, pointing to hybrid analogue - digital devices as a path to scaling networks, and provides insights into the dynamics of adaptive optimizers.