# Nonlinear Optimization with GPU-Accelerated Neural Network Constraints

arXiv: https://arxiv.org/abs/2509.22462 · HTML: https://mlsanity.com/paper/2509.22462/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 26 Sep 2025.

## Builds on (most cited first, 16 of 16)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Efficient Neural Network Robustness Certification with General Activation Functions](https://mlsanity.com/paper/1811.00866/) (2018)
- [JuMP 1.0: Recent improvements to a modeling language for mathematical optimization](https://mlsanity.com/paper/2206.03866/) (2022)
- [OMLT: Optimization & Machine Learning Toolkit](https://mlsanity.com/paper/2202.02414/) (2022)
- [PowerModels.jl: An Open-Source Framework for Exploring Power Flow Formulations](https://mlsanity.com/paper/1711.01728/) (2017)
- [Global Deterministic Optimization with Artificial Neural Networks Embedded](https://mlsanity.com/paper/1801.07114/) (2018)
- [Practical Large-Scale Linear Programming using Primal-Dual Hybrid Gradient](https://mlsanity.com/paper/2106.04756/) (2021)
- [Accelerating Optimal Power Flow with GPUs: SIMD Abstraction of Nonlinear Programs and Condensed-Space Interior-Point Methods](https://mlsanity.com/paper/2307.16830/) (2023)
- [cuPDLP.jl: A GPU Implementation of Restarted Primal-Dual Hybrid Gradient for Linear Programming in Julia](https://mlsanity.com/paper/2311.12180/) (2023)
- [Recent Developments in Security-Constrained AC Optimal Power Flow: Overview of Challenge 1 in the ARPA-E Grid Optimization Competition](https://mlsanity.com/paper/2206.07843/) (2022)
- [A Comparison of Strategies to Embed Physics-Informed Neural Networks in Nonlinear Model Predictive Control Formulations Solved via Direct Transcription](https://mlsanity.com/paper/2501.06335/) (2025)
- [GPU-accelerated dynamic nonlinear optimization with ExaModels and MadNLP](https://mlsanity.com/paper/2403.15913/) (2024)
- [Process flowsheet optimization with surrogate and implicit formulations of a Gibbs reactor](https://mlsanity.com/paper/2310.09307/) (2023)
- [Transient Stability-Constrained OPF: Neural Network Surrogate Models and Pricing Stability](https://mlsanity.com/paper/2502.01844/) (2025)
- [A Feasible Reduced Space Method for Real-Time Optimal Power Flow](https://mlsanity.com/paper/2110.02590/) (2021)
- [Variable aggregation for nonlinear optimization problems](https://mlsanity.com/paper/2502.13869/) (2025)

## Built on by (most cited first, 1 of 1)

- [Generating adversarial inputs for a graph neural network model of AC power flow](https://mlsanity.com/paper/2602.17975/) (2026)

## Abstract (arXiv)

We propose a reduced-space formulation for optimizing over trained neural networks where the network's outputs and derivatives are evaluated on a GPU. To do this, we treat the neural network as a "gray box" where intermediate variables and constraints are not exposed to the optimization solver. Compared to the full-space formulation, in which intermediate variables and constraints are exposed to the optimization solver, the reduced-space formulation leads to faster solves and fewer iterations in an interior point method. We demonstrate the benefits of this method on two optimization problems: Adversarial generation for a classifier trained on MNIST images and security-constrained optimal power flow with transient feasibility enforced using a neural network surrogate.