mlsanityData to 14 Oct 2026

BOND: License to Train with Black-Box Functions

arXiv 2510.25074 ↗First posted 29 Oct 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce Bounded Numerical Differentiation (BOND), a perturbative method for estimating the gradients of black-box functions. BOND is distinguished by its formulation, which adaptively bounds perturbations to ensure accurate sign estimation, and by its implementation, which operates at black-box interfaces. This enables BOND to be more accurate and scalable compared to existing methods, facilitating end-to-end training of architectures that incorporate non-autodifferentiable modules. We observe that these modules, implemented in our experiments as frozen networks, can enhance model performance without increasing the number of trainable parameters. Our findings highlight the potential of leveraging fixed transformations to expand model capacity, pointing to hybrid analogue - digital devices as a path to scaling networks, and provides insights into the dynamics of adaptive optimizers.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 56 references that are in the map. First posted on arXiv 29 Oct 2025.

Builds on

18 of 56 references.

References · 18

  1. 2016Densely Connected Convolutional Networkscited by 6,720
  2. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 666
  3. 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 501
  4. 2013Optimal rates for zero-order convex optimization: the power of two function evaluationscited by 198
  5. 2020A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learningcited by 162
  6. 2021Next Generation Reservoir Computingcited by 160
  7. 2019ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimizationcited by 76
  8. 2017Stochastic Zeroth-order Optimization in High Dimensionscited by 63
  9. 2015Equilibrated adaptive learning rates for non-convex optimizationcited by 42
  10. 2021Strength of Minibatch Noise in SGDcited by 39
  11. 2023A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learningcited by 22
  12. 2023Analogue and Physical Reservoir Computing Using Water Wavescited by 16
6 more
  1. 2020A New One-Point Residual-Feedback Oracle For Black-Box Learning and Controlcited by 14
  2. 2023Multiplexed gradient descent: Fast online training of modern datasets on hardware neural networks without backpropagationcited by 8
  3. 2018Stochastic Zeroth-order Optimization via Variance Reduction methodcited by 5
  4. 2025Scaling Recurrent Neural Networks to a Billion Parameters with Zero-Order Optimizationcited by 4
  5. 2015Convergence Rates of Finite Difference Stochastic Approximation Algorithmscited by 2
  6. 2025Perturbative Gradient Training: A novel training paradigm for bridging the gap between deep neural networks and physical reservoir computingcited by 1

Built on by

Not cited yet.