# Optimization over Trained Neural Networks: Going Large with Gradient-Based Algorithms

arXiv: https://arxiv.org/abs/2512.24295 · HTML: https://mlsanity.com/paper/2512.24295/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Dec 2025.

## Built on by (most cited first, 2 of 2)

- [Relaxation-Informed Training of Neural Network Surrogate Models](https://mlsanity.com/paper/2604.22746/) (2026)
- [Monotone Optimisation with Learned Projections](https://mlsanity.com/paper/2601.20983/) (2026)

## Abstract (arXiv)

When optimizing a nonlinear objective, one can employ a neural network as a surrogate for the nonlinear function. However, the resulting optimization model can be time-consuming to solve globally with exact methods. As a result, local search that exploits the neural-network structure has been employed to find good solutions within a reasonable time limit. For such methods, a lower per-iteration cost is advantageous when solving larger models. The contribution of this paper is two-fold. First, we propose a gradient-based algorithm with lower per-iteration cost than existing methods. Second, we further adapt this algorithm to exploit the piecewise-linear structure of neural networks that use Rectified Linear Units (ReLUs). In line with prior research, our methods become competitive with -- and then dominant over -- other local search methods as the optimization models become larger.