# HardNet: Hard-Constrained Neural Networks with Universal Approximation Guarantees

arXiv: https://arxiv.org/abs/2410.10807 · HTML: https://mlsanity.com/paper/2410.10807/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 71 references that are in the map.
- First posted on arXiv 14 Oct 2024.

## Builds on (most cited first, 30 of 53)

- [Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks](https://mlsanity.com/paper/1702.01135/) (2017)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Neural Probabilistic Logic Programming in DeepProbLog](https://mlsanity.com/paper/1907.08194/) (2019)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [A Semantic Loss Function for Deep Learning with Symbolic Knowledge](https://mlsanity.com/paper/1711.11157/) (2017)
- [Formal Verification of Piece-Wise Linear Feed-Forward Neural Networks](https://mlsanity.com/paper/1705.01320/) (2017)
- [Algorithms for Verifying Deep Neural Networks](https://mlsanity.com/paper/1903.06758/) (2019)
- [Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation](https://mlsanity.com/paper/2112.05124/) (2021)
- [Universal Approximation with Deep Narrow Networks](https://mlsanity.com/paper/1905.08539/) (2019)
- [Logic Tensor Networks](https://mlsanity.com/paper/2012.13635/) (2020)
- [DC3: A learning method for optimization with hard constraints](https://mlsanity.com/paper/2104.12225/) (2021)
- [Verifiable Reinforcement Learning via Policy Extraction](https://mlsanity.com/paper/1805.08328/) (2018)
- [Approximating Continuous Functions by ReLU Nets of Minimal Width](https://mlsanity.com/paper/1710.11278/) (2017)
- [Learning Stable Deep Dynamics Models](https://mlsanity.com/paper/2001.06116/) (2020)
- [Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and Segmentation](https://mlsanity.com/paper/1705.08302/) (2017)
- [Semantic Probabilistic Layers for Neuro-Symbolic Learning](https://mlsanity.com/paper/2206.00426/) (2022)
- [Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming](https://mlsanity.com/paper/1903.01287/) (2019)
- [Constrained Convolutional Neural Networks for Weakly Supervised Segmentation](https://mlsanity.com/paper/1506.03648/) (2015)
- [Minimum Width for Universal Approximation](https://mlsanity.com/paper/2006.08859/) (2020)
- [An Abstraction-Based Framework for Neural Network Verification](https://mlsanity.com/paper/1910.14574/) (2019)
- [Coherent Hierarchical Multi-Label Classification Networks](https://mlsanity.com/paper/2010.10151/) (2020)
- [Introduction to Neural Network Verification](https://mlsanity.com/paper/2109.10317/) (2021)
- [Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learning](https://mlsanity.com/paper/2206.08321/) (2022)
- [A-NeSI: A Scalable Approximate Method for Probabilistic Neurosymbolic Inference](https://mlsanity.com/paper/2212.12393/) (2022)
- [Neurosymbolic Reinforcement Learning with Formally Verified Exploration](https://mlsanity.com/paper/2009.12612/) (2020)
- [Probably Approximately Correct Constrained Learning](https://mlsanity.com/paper/2006.05487/) (2020)
- [Equivariant Transporter Network](https://mlsanity.com/paper/2202.09400/) (2022)
- [Imposing Hard Constraints on Deep Networks: Promises and Limitations](https://mlsanity.com/paper/1706.02025/) (2017)

## Built on by (most cited first, 30 of 31)

- [ENFORCE: Nonlinear Constrained Learning with Adaptive-depth Neural Projection](https://mlsanity.com/paper/2502.06774/) (2025)
- [Position: Adopt Constraints Over Fixed Penalties in Deep Learning](https://mlsanity.com/paper/2505.20628/) (2025)
- [Physics-Constrained Machine Learning for Chemical Engineering](https://mlsanity.com/paper/2508.20649/) (2025)
- [LMI-Net: Linear Matrix Inequality--Constrained Neural Networks via Differentiable Projection Layers](https://mlsanity.com/paper/2604.05374/) (2026)
- [Soft-Radial Projection for Constrained End-to-End Learning](https://mlsanity.com/paper/2602.03461/) (2026)
- [Improving Feasibility via Fast Autoencoder-Based Projections](https://mlsanity.com/paper/2604.03489/) (2026)
- [ECO: Energy-Constrained Operator Learning for Chaotic Dynamics with Boundedness Guarantees](https://mlsanity.com/paper/2512.01984/) (2025)
- [SnareNet: Flexible Repair Layers for Neural Networks with Hard Constraints](https://mlsanity.com/paper/2602.09317/) (2026)
- [HardNet++: Nonlinear Constraint Enforcement in Neural Networks](https://mlsanity.com/paper/2604.19669/) (2026)
- [HoP: Homeomorphic Polar Learning for Hard Constrained Optimization](https://mlsanity.com/paper/2502.00304/) (2025)
- [Constrained Online Decision-Making: A Unified Framework](https://mlsanity.com/paper/2505.07101/) (2025)
- [VFSI: Validity First Spatial Intelligence for Constraint-Guided Traffic Diffusion](https://mlsanity.com/paper/2509.23971/) (2025)
- [AdamFLIP: Adaptive Momentum Feedback Linearization Optimization for Hard Constrained PINN Training](https://mlsanity.com/paper/2605.08408/) (2026)
- [Exact Fixed-Point Constraints in Neural-ODEs with Provable Universality](https://mlsanity.com/paper/2605.10613/) (2026)
- [DiffSlack: Learning under Nonlinear Inequality Constraints via Learnable Slack Variables](https://mlsanity.com/paper/2606.05247/) (2026)
- [LEAF: A Learning-Enabled ADMM Framework for Accelerated Convex Optimization](https://mlsanity.com/paper/2606.08993/) (2026)
- [The Standard Interpretable Model: A general theory of interpretable machine learning to deductively design interpretable methods using Lagrangian mechanics](https://mlsanity.com/paper/2606.12289/) (2026)
- [Provably Safe, Yet Scalable Reinforcement Learning](https://mlsanity.com/paper/2606.14536/) (2026)
- [Self-Certifying Primal-Dual Optimization Proxies for Large-Scale Batch Economic Dispatch](https://mlsanity.com/paper/2510.15850/) (2025)
- [Constrained Policy Optimization via Sampling-Based Weight-Space Projection](https://mlsanity.com/paper/2512.13788/) (2025)
- [Geometry-Preserving Neural Architectures on Manifolds with Boundary](https://mlsanity.com/paper/2602.03082/) (2026)
- [Chimera: Neuro-Symbolic Attention Primitives for Trustworthy Dataplane Intelligence](https://mlsanity.com/paper/2602.12851/) (2026)
- [Stochastic Dimension Implicit Functional Projections for Global Integral Conservation in High-Dimensional PINNs](https://mlsanity.com/paper/2603.29237/) (2026)
- [CAffNet: Hard Constraint-Affine Neural Networks](https://mlsanity.com/paper/2605.24437/) (2026)
- [Neural Slack Variables for Shape Constraints](https://mlsanity.com/paper/2606.13803/) (2026)
- [ShardNet: Training Neural Controllers with Hard, Non-Convex Constraints](https://mlsanity.com/paper/2606.30935/) (2026)
- [End-to-End Learning of Safe Optimal Feedback Control in High Dimensions with Control Barrier Function Layers](https://mlsanity.com/paper/2607.20674/) (2026)
- [CG4AI: A Column Generation Framework for Training AI Models Under Constraints](https://mlsanity.com/paper/2608.26375/) (2026)
- [Proximal Residual Value Functions for Consistent Planning and Real-Time Execution](https://mlsanity.com/paper/2609.23242/) (2026)
- [AT-SKM-Net: An Accelerated Trainable Sampling Kaczmarz-Motzkin Framework for Linear Hard-Constraint Feasibility on Dynamic Graphs](https://mlsanity.com/paper/2609.30088/) (2026)

## Abstract (arXiv)

Incorporating prior knowledge or specifications of input-output relationships into machine learning models has attracted significant attention, as it enhances generalization from limited data and yields conforming outputs. However, most existing approaches use soft constraints by penalizing violations through regularization, which offers no guarantee of constraint satisfaction, especially on inputs far from the training distribution--an essential requirement in safety-critical applications. On the other hand, imposing hard constraints on neural networks may hinder their representational power, adversely affecting performance. To address this, we propose HardNet, a practical framework for constructing neural networks that inherently satisfy hard constraints without sacrificing model capacity. Unlike approaches that modify outputs only at inference time, HardNet enables end-to-end training with hard constraint guarantees, leading to improved performance. To the best of our knowledge, HardNet is the first method that enables efficient and differentiable enforcement of more than one input-dependent inequality constraint. It allows unconstrained optimization of the network parameters using standard algorithms by appending a differentiable closed-form enforcement layer to the network's output. Furthermore, we show that HardNet retains neural networks' universal approximation capabilities. We demonstrate its versatility and effectiveness across various applications: learning with piecewise constraints, learning optimization solvers with guaranteed feasibility, and optimizing control policies in safety-critical systems.