# Imposing Hard Constraints on Deep Networks: Promises and Limitations

arXiv: https://arxiv.org/abs/1706.02025 · HTML: https://mlsanity.com/paper/1706.02025/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2017.

## Built on by (most cited first, 30 of 43)

- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [Neural Networks with Physics-Informed Architectures and Constraints for Dynamical Systems Modeling](https://mlsanity.com/paper/2109.06407/) (2021)
- [Enhanced Physics-Informed Neural Networks with Augmented Lagrangian Relaxation Method (AL-PINNs)](https://mlsanity.com/paper/2205.01059/) (2022)
- [HardNet: Hard-Constrained Neural Networks with Universal Approximation Guarantees](https://mlsanity.com/paper/2410.10807/) (2024)
- [RAYEN: Imposition of Hard Convex Constraints on Neural Networks](https://mlsanity.com/paper/2307.08336/) (2023)
- [Robust Calibration of Large Vision-Language Adapters](https://mlsanity.com/paper/2407.13588/) (2024)
- [Random-Set Neural Networks (RS-NN)](https://mlsanity.com/paper/2307.05772/) (2023)
- [Pinet: Optimizing hard-constrained neural networks with orthogonal projection layers](https://mlsanity.com/paper/2508.10480/) (2025)
- [HANNA: Hard-constraint Neural Network for Consistent Activity Coefficient Prediction](https://mlsanity.com/paper/2407.18011/) (2024)
- [Physics-Informed Neural Networks with Hard Nonlinear Equality and Inequality Constraints](https://mlsanity.com/paper/2507.08124/) (2025)
- [A New Computationally Simple Approach for Implementing Neural Networks with Output Hard Constraints](https://mlsanity.com/paper/2307.10459/) (2023)
- [Deep Learning Explicit Differentiable Predictive Control Laws for Buildings](https://mlsanity.com/paper/2107.11843/) (2021)
- [Neural network training under semidefinite constraints](https://mlsanity.com/paper/2201.00632/) (2022)
- [Physics-informed linear regression is competitive with two Machine Learning methods in residential building MPC](https://mlsanity.com/paper/2110.15911/) (2021)
- [Learning to Optimize for Mixed-Integer Non-linear Programming with Feasibility Guarantees](https://mlsanity.com/paper/2410.11061/) (2024)
- [Guaranteed Conformance of Neurosymbolic Models to Natural Constraints](https://mlsanity.com/paper/2212.01346/) (2022)
- [Lagrangian dual framework for conservative neural network solutions of kinetic equations](https://mlsanity.com/paper/2106.12147/) (2021)
- [HybridSDF: Combining Deep Implicit Shapes and Geometric Primitives for 3D Shape Representation and Manipulation](https://mlsanity.com/paper/2109.10767/) (2021)
- [Fast Jacobian-Vector Product for Deep Networks](https://mlsanity.com/paper/2104.00219/) (2021)
- [Calabi-Yau metrics through Grassmannian learning and Donaldson's algorithm](https://mlsanity.com/paper/2410.11284/) (2024)
- [DAE-HardNet: A Physics Constrained Neural Network Enforcing Differential-Algebraic Hard Constraints](https://mlsanity.com/paper/2512.05881/) (2025)
- [Soft-Radial Projection for Constrained End-to-End Learning](https://mlsanity.com/paper/2602.03461/) (2026)
- [Leveraging Unlabeled Data for Entity-Relation Extraction through Probabilistic Constraint Satisfaction](https://mlsanity.com/paper/2103.11062/) (2021)
- [Hierarchical learning, forecasting coherent spatio-temporal individual and aggregated building loads](https://mlsanity.com/paper/2301.12967/) (2023)
- [Class and Region-Adaptive Constraints for Network Calibration](https://mlsanity.com/paper/2403.12364/) (2024)
- [SnareNet: Flexible Repair Layers for Neural Networks with Hard Constraints](https://mlsanity.com/paper/2602.09317/) (2026)
- [Towards Positive Jacobian: Learn to Postprocess Diffeomorphic Image Registration with Matrix Exponential](https://mlsanity.com/paper/2202.00749/) (2022)
- [Constrained Empirical Risk Minimization: Theory and Practice](https://mlsanity.com/paper/2302.04729/) (2023)
- [Implicit Bias in Matrix Factorization and its Explicit Realization in a New Architecture](https://mlsanity.com/paper/2501.16322/) (2025)
- [BEKAN: Boundary condition-guaranteed evolutionary Kolmogorov-Arnold networks with radial basis functions for solving PDE problems](https://mlsanity.com/paper/2510.03576/) (2025)

## Abstract (arXiv)

Imposing constraints on the output of a Deep Neural Net is one way to improve the quality of its predictions while loosening the requirements for labeled training data. Such constraints are usually imposed as soft constraints by adding new terms to the loss function that is minimized during training. An alternative is to impose them as hard constraints, which has a number of theoretical benefits but has not been explored so far due to the perceived intractability of the problem. In this paper, we show that imposing hard constraints can in fact be done in a computationally feasible way and delivers reasonable results. However, the theoretical benefits do not materialize and the resulting technique is no better than existing ones relying on soft constraints. We analyze the reasons for this and hope to spur other researchers into proposing better solutions.