mlsanityData to 7 Oct 2026

ReLU Networks as Surrogate Models in Mixed-Integer Linear Programs

arXiv 1907.03140 ↗First posted 6 Jul 2019math.OC cs.LGcited by 33

Not on the map. Why?

Abstract (arXiv)

We consider the embedding of piecewise-linear deep neural networks (ReLU networks) as surrogate models in mixed-integer linear programming (MILP) problems. A MILP formulation of ReLU networks has recently been applied by many authors to probe for various model properties subject to input bounds. The formulation is obtained by programming each ReLU operator with a binary variable and applying the big-M method. The efficiency of the formulation hinges on the tightness of the bounds defined by the big-M values. When ReLU networks are embedded in a larger optimization problem, the presence of output bounds can be exploited in bound tightening. To this end, we devise and study several bound tightening procedures that consider both input and output bounds. Our numerical results show that bound tightening may reduce solution times considerably, and that small-sized ReLU networks are suitable as surrogate models in mixed-integer linear programs.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 7; 2023: 7; 2024: 2; 2025: 4; 2026: 5.

33 papers.

Sort:

  1. 2021Partition-based formulations for mixed-integer optimization of trained ReLU neural networkscited by 39
  2. 2023When Deep Learning Meets Polyhedral Theory: A Surveycited by 39
  3. 2021Mixed-Integer Optimization with Constraint Learningcited by 30
  4. 2024Physics-Informed Neural Networks with Hard Linear Equality Constraintscited by 30
  5. 2021Optimization with Constraint Learning: A Framework and Surveycited by 28
  6. 2023Finding Regions of Counterfactual Explanations via Robust Optimizationcited by 19
  7. 2021Modeling the AC Power Flow Equations with Optimally Compact Neural Networks: Application to Unit Commitmentcited by 17
  8. 2022Neur2SP: Neural Two-Stage Stochastic Programmingcited by 16
  9. 2021Between steps: Intermediate relaxations between big-M and convex hull formulationscited by 11
  10. 2022Closing the Loop: A Framework for Trustworthy Machine Learning in Power Systemscited by 9
  11. 2021Modeling Design and Control Problems Involving Neural Network Surrogatescited by 8
  12. 2022Counterfactual Explanations Using Optimization With Constraint Learningcited by 8
21 more
  1. 2023PySCIPOpt-ML: Embedding Trained Machine Learning Models into Mixed-Integer Programscited by 8
  2. 2024Generative AI and Process Systems Engineering: The Next Frontiercited by 8
  3. 2022P-split formulations: A class of intermediate formulations between big-M and convex hull for disjunctive constraintscited by 7
  4. 2023Global Optimization: A Machine Learning Approachcited by 4
  5. 2022On data-driven chance constraint learning for mixed-integer optimization problemscited by 2
  6. 2023Model-based feature selection for neural networks: A mixed-integer programming approachcited by 2
  7. 2023Deep Learning for Two-Stage Robust Integer Optimizationcited by 2
  8. 2025An analysis of optimization problems involving ReLU neural networkscited by 2
  9. 2021Fast Design Space Exploration of Nonlinear Systems: Part Icited by 1
  10. 2021Optimizing over an ensemble of neural networkscited by 1
  11. 2022Constraint Learning to Define Trust Regions in Predictive-Model Embedded Optimizationcited by 1
  12. 2023Physics Informed Piecewise Linear Neural Networks for Process Optimizationcited by 1
  13. 2026Relaxation-Informed Training of Neural Network Surrogate Modelscited by 1
  14. 2022Emission-Aware Optimization of Gas Networks: Input-Convex Neural Network Approachcited by 0
  15. 2025Optimization over Trained (and Sparse) Neural Networks: A Surrogate within a Surrogatecited by 0
  16. 2025Conformal Mixed-Integer Constraint Learning with Feasibility Guaranteescited by 0
  17. 2025Neural Two-Stage Stochastic Optimization for Solving Unit Commitment Problemcited by 0
  18. 2026Efficient reformulations of ReLU deep neural networks for surrogate modelling in power system optimisationcited by 0
  19. 2026An Efficient Spatial Branch-and-Bound Algorithm for Global Optimization of Gaussian Process Posterior Mean Functionscited by 0
  20. 2026Input convex neural networks as surrogates in mathematical optimisationcited by 0
  21. 2026DCEmbed: Scalable Optimization over Neural Surrogatescited by 0