# A New Computationally Simple Approach for Implementing Neural Networks with Output Hard Constraints

arXiv: https://arxiv.org/abs/2307.10459 · HTML: https://mlsanity.com/paper/2307.10459/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 19 Jul 2023.

## Builds on (most cited first, 15 of 15)

- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [DC3: A learning method for optimization with hard constraints](https://mlsanity.com/paper/2104.12225/) (2021)
- [OSQP: An Operator Splitting Solver for Quadratic Programs](https://mlsanity.com/paper/1711.08013/) (2017)
- [End-to-End Constrained Optimization Learning: A Survey](https://mlsanity.com/paper/2103.16378/) (2021)
- [Imposing Hard Constraints on Deep Networks: Promises and Limitations](https://mlsanity.com/paper/1706.02025/) (2017)
- [Learning differentiable solvers for systems with hard constraints](https://mlsanity.com/paper/2207.08675/) (2022)
- [Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method](https://mlsanity.com/paper/2012.06148/) (2020)
- [Learning to Solve Optimization Problems with Hard Linear Constraints](https://mlsanity.com/paper/2208.10611/) (2022)
- [Learning Hard Optimization Problems: A Data Generation Perspective](https://mlsanity.com/paper/2106.02601/) (2021)
- [POLICE: Provably Optimal Linear Constraint Enforcement for Deep Neural Networks](https://mlsanity.com/paper/2211.01340/) (2022)
- [Linearly Constrained Neural Networks](https://mlsanity.com/paper/2002.01600/) (2020)
- [Teaching the Old Dog New Tricks: Supervised Learning with Constraints](https://mlsanity.com/paper/2002.10766/) (2020)
- [Sample-Specific Output Constraints for Neural Networks](https://mlsanity.com/paper/2003.10258/) (2020)
- [Iterative Supervised Learning for Regression with Constraints](https://mlsanity.com/paper/2201.06529/) (2022)

## Built on by (most cited first, 12 of 12)

- [HardNet: Hard-Constrained Neural Networks with Universal Approximation Guarantees](https://mlsanity.com/paper/2410.10807/) (2024)
- [FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guarantees](https://mlsanity.com/paper/2506.00362/) (2025)
- [ENFORCE: Nonlinear Constrained Learning with Adaptive-depth Neural Projection](https://mlsanity.com/paper/2502.06774/) (2025)
- [Learning Constrained Optimization with Deep Augmented Lagrangian Methods](https://mlsanity.com/paper/2403.03454/) (2024)
- [Dual Lagrangian Learning for Conic Optimization](https://mlsanity.com/paper/2402.03086/) (2024)
- [Metric Learning to Accelerate Convergence of Operator Splitting Methods for Differentiable Parametric Programming](https://mlsanity.com/paper/2404.00882/) (2024)
- [Incorporating Expert Rules into Neural Networks in the Framework of Concept-Based Learning](https://mlsanity.com/paper/2402.14726/) (2024)
- [Soft-Radial Projection for Constrained End-to-End Learning](https://mlsanity.com/paper/2602.03461/) (2026)
- [Enforcing convex constraints in Graph Neural Networks](https://mlsanity.com/paper/2510.11227/) (2025)
- [Learning to Solve Constrained Bilevel Control Co-Design Problems](https://mlsanity.com/paper/2507.09050/) (2025)
- [ShardNet: Training Neural Controllers with Hard, Non-Convex Constraints](https://mlsanity.com/paper/2606.30935/) (2026)
- [AT-SKM-Net: An Accelerated Trainable Sampling Kaczmarz-Motzkin Framework for Linear Hard-Constraint Feasibility on Dynamic Graphs](https://mlsanity.com/paper/2609.30088/) (2026)

## Abstract (arXiv)

A new computationally simple method of imposing hard convex constraints on the neural network output values is proposed. The key idea behind the method is to map a vector of hidden parameters of the network to a point that is guaranteed to be inside the feasible set defined by a set of constraints. The mapping is implemented by the additional neural network layer with constraints for output. The proposed method is simply extended to the case when constraints are imposed not only on the output vectors, but also on joint constraints depending on inputs. The projection approach to imposing constraints on outputs can simply be implemented in the framework of the proposed method. It is shown how to incorporate different types of constraints into the proposed method, including linear and quadratic constraints, equality constraints, and dynamic constraints, constraints in the form of boundaries. An important feature of the method is its computational simplicity. Complexities of the forward pass of the proposed neural network layer by linear and quadratic constraints are O(n*m) and O(n^2*m), respectively, where n is the number of variables, m is the number of constraints. Numerical experiments illustrate the method by solving optimization and classification problems. The code implementing the method is publicly available.