# ReLU Networks as Surrogate Models in Mixed-Integer Linear Programs

arXiv: https://arxiv.org/abs/1907.03140 · HTML: https://mlsanity.com/paper/1907.03140/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2019.

## Built on by (most cited first, 30 of 33)

- [Partition-based formulations for mixed-integer optimization of trained ReLU neural networks](https://mlsanity.com/paper/2102.04373/) (2021)
- [When Deep Learning Meets Polyhedral Theory: A Survey](https://mlsanity.com/paper/2305.00241/) (2023)
- [Mixed-Integer Optimization with Constraint Learning](https://mlsanity.com/paper/2111.04469/) (2021)
- [Physics-Informed Neural Networks with Hard Linear Equality Constraints](https://mlsanity.com/paper/2402.07251/) (2024)
- [Optimization with Constraint Learning: A Framework and Survey](https://mlsanity.com/paper/2110.02121/) (2021)
- [Finding Regions of Counterfactual Explanations via Robust Optimization](https://mlsanity.com/paper/2301.11113/) (2023)
- [Modeling the AC Power Flow Equations with Optimally Compact Neural Networks: Application to Unit Commitment](https://mlsanity.com/paper/2110.11269/) (2021)
- [Neur2SP: Neural Two-Stage Stochastic Programming](https://mlsanity.com/paper/2205.12006/) (2022)
- [Between steps: Intermediate relaxations between big-M and convex hull formulations](https://mlsanity.com/paper/2101.12708/) (2021)
- [Modeling Design and Control Problems Involving Neural Network Surrogates](https://mlsanity.com/paper/2111.10489/) (2021)
- [Closing the Loop: A Framework for Trustworthy Machine Learning in Power Systems](https://mlsanity.com/paper/2203.07505/) (2022)
- [Generative AI and Process Systems Engineering: The Next Frontier](https://mlsanity.com/paper/2402.10977/) (2024)
- [Counterfactual Explanations Using Optimization With Constraint Learning](https://mlsanity.com/paper/2209.10997/) (2022)
- [PySCIPOpt-ML: Embedding Trained Machine Learning Models into Mixed-Integer Programs](https://mlsanity.com/paper/2312.08074/) (2023)
- [P-split formulations: A class of intermediate formulations between big-M and convex hull for disjunctive constraints](https://mlsanity.com/paper/2202.05198/) (2022)
- [Global Optimization: A Machine Learning Approach](https://mlsanity.com/paper/2311.01742/) (2023)
- [On data-driven chance constraint learning for mixed-integer optimization problems](https://mlsanity.com/paper/2207.03844/) (2022)
- [Model-based feature selection for neural networks: A mixed-integer programming approach](https://mlsanity.com/paper/2302.10344/) (2023)
- [Deep Learning for Two-Stage Robust Integer Optimization](https://mlsanity.com/paper/2310.04345/) (2023)
- [An analysis of optimization problems involving ReLU neural networks](https://mlsanity.com/paper/2502.03016/) (2025)
- [Relaxation-Informed Training of Neural Network Surrogate Models](https://mlsanity.com/paper/2604.22746/) (2026)
- [Fast Design Space Exploration of Nonlinear Systems: Part I](https://mlsanity.com/paper/2104.01747/) (2021)
- [Optimizing over an ensemble of neural networks](https://mlsanity.com/paper/2112.07007/) (2021)
- [Constraint Learning to Define Trust Regions in Predictive-Model Embedded Optimization](https://mlsanity.com/paper/2201.04429/) (2022)
- [Physics Informed Piecewise Linear Neural Networks for Process Optimization](https://mlsanity.com/paper/2302.00990/) (2023)
- [Conformal Mixed-Integer Constraint Learning with Feasibility Guarantees](https://mlsanity.com/paper/2506.03531/) (2025)
- [Emission-Aware Optimization of Gas Networks: Input-Convex Neural Network Approach](https://mlsanity.com/paper/2209.08645/) (2022)
- [Optimization over Trained (and Sparse) Neural Networks: A Surrogate within a Surrogate](https://mlsanity.com/paper/2505.01985/) (2025)
- [Neural Two-Stage Stochastic Optimization for Solving Unit Commitment Problem](https://mlsanity.com/paper/2507.09503/) (2025)
- [Efficient reformulations of ReLU deep neural networks for surrogate modelling in power system optimisation](https://mlsanity.com/paper/2601.14673/) (2026)

## Abstract (arXiv)

We consider the embedding of piecewise-linear deep neural networks (ReLU networks) as surrogate models in mixed-integer linear programming (MILP) problems. A MILP formulation of ReLU networks has recently been applied by many authors to probe for various model properties subject to input bounds. The formulation is obtained by programming each ReLU operator with a binary variable and applying the big-M method. The efficiency of the formulation hinges on the tightness of the bounds defined by the big-M values. When ReLU networks are embedded in a larger optimization problem, the presence of output bounds can be exploited in bound tightening. To this end, we devise and study several bound tightening procedures that consider both input and output bounds. Our numerical results show that bound tightening may reduce solution times considerably, and that small-sized ReLU networks are suitable as surrogate models in mixed-integer linear programs.