# Optimization-Informed Neural Networks

arXiv: https://arxiv.org/abs/2210.02113 · HTML: https://mlsanity.com/paper/2210.02113/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 65 references that are in the map.
- First posted on arXiv 5 Oct 2022.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Combinatorial optimization and reasoning with graph neural networks](https://mlsanity.com/paper/2102.09544/) (2021)
- [Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations](https://mlsanity.com/paper/1706.04702/) (2017)
- [Solving Mixed Integer Programs Using Neural Networks](https://mlsanity.com/paper/2012.13349/) (2020)
- [An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications](https://mlsanity.com/paper/1908.10407/) (2019)
- [Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations](https://mlsanity.com/paper/1709.05963/) (2017)
- [Neural Network with Unbounded Activation Functions is Universal Approximator](https://mlsanity.com/paper/1505.03654/) (2015)
- [Deep Learning in Bioinformatics](https://mlsanity.com/paper/1603.06430/) (2016)
- [Hamiltonian neural networks for solving equations of motion](https://mlsanity.com/paper/2001.11107/) (2020)
- [Solving Differential Equations Using Neural Network Solution Bundles](https://mlsanity.com/paper/2006.14372/) (2020)
- [A Deep Collocation Method for the Bending Analysis of Kirchhoff Plate](https://mlsanity.com/paper/2102.02617/) (2021)
- [A variational inequality framework for network games: Existence, uniqueness, convergence and sensitivity analysis](https://mlsanity.com/paper/1712.08277/) (2017)

## Abstract (arXiv)

Solving constrained nonlinear optimization problems (CNLPs) is a longstanding problem that arises in various fields, e.g., economics, computer science, and engineering. We propose optimization-informed neural networks (OINN), a deep learning approach to solve CNLPs. By neurodynamic optimization methods, a CNLP is first reformulated as an initial value problem (IVP) involving an ordinary differential equation (ODE) system. A neural network model is then used as an approximate solution for this IVP, with the endpoint being the prediction to the CNLP. We propose a novel training algorithm that directs the model to hold the best prediction during training. In a nutshell, OINN transforms a CNLP into a neural network training problem. By doing so, we can solve CNLPs based on deep learning infrastructure only, without using standard optimization solvers or numerical integration solvers. The effectiveness of the proposed approach is demonstrated through a collection of classical problems, e.g., variational inequalities, nonlinear complementary problems, and standard CNLPs.