# RAYEN: Imposition of Hard Convex Constraints on Neural Networks

arXiv: https://arxiv.org/abs/2307.08336 · HTML: https://mlsanity.com/paper/2307.08336/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 96 references that are in the map.
- First posted on arXiv 17 Jul 2023.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Differentiable MPC for End-to-end Planning and Control](https://mlsanity.com/paper/1810.13400/) (2018)
- [DC3: A learning method for optimization with hard constraints](https://mlsanity.com/paper/2104.12225/) (2021)
- [Trivializations for Gradient-Based Optimization on Manifolds](https://mlsanity.com/paper/1909.09501/) (2019)
- [Differentiating Through a Cone Program](https://mlsanity.com/paper/1904.09043/) (2019)
- [Eigenvalue and Generalized Eigenvalue Problems: Tutorial](https://mlsanity.com/paper/1903.11240/) (2019)
- [Imposing Hard Constraints on Deep Networks: Promises and Limitations](https://mlsanity.com/paper/1706.02025/) (2017)
- [Motion Planning around Obstacles with Convex Optimization](https://mlsanity.com/paper/2205.04422/) (2022)
- [Learning to Solve Optimization Problems with Hard Linear Constraints](https://mlsanity.com/paper/2208.10611/) (2022)
- [Learning Arm-Assisted Fall Damage Reduction and Recovery for Legged Mobile Manipulators](https://mlsanity.com/paper/2303.05486/) (2023)
- [Feedback MPC for Torque-Controlled Legged Robots](https://mlsanity.com/paper/1905.06144/) (2019)
- [MADER: Trajectory Planner in Multi-Agent and Dynamic Environments](https://mlsanity.com/paper/2010.11061/) (2020)
- [Convex Relaxations for Pose Graph Optimization with Outliers](https://mlsanity.com/paper/1801.02112/) (2018)
- [POLICE: Provably Optimal Linear Constraint Enforcement for Deep Neural Networks](https://mlsanity.com/paper/2211.01340/) (2022)
- [Linearly Constrained Neural Networks](https://mlsanity.com/paper/2002.01600/) (2020)
- [Distributed Certifiably Correct Pose-Graph Optimization](https://mlsanity.com/paper/1911.03721/) (2019)
- [FASTER: Fast and Safe Trajectory Planner for Navigation in Unknown Environments](https://mlsanity.com/paper/2001.04420/) (2020)
- [PDP: A General Neural Framework for Learning Constraint Satisfaction Solvers](https://mlsanity.com/paper/1903.01969/) (2019)
- [Neural network training under semidefinite constraints](https://mlsanity.com/paper/2201.00632/) (2022)
- [Teach-Repeat-Replan: A Complete and Robust System for Aggressive Flight in Complex Environments](https://mlsanity.com/paper/1907.00520/) (2019)
- [MINVO Basis: Finding Simplexes with Minimum Volume Enclosing Polynomial Curves](https://mlsanity.com/paper/2010.10726/) (2020)
- [A Linearly Constrained Nonparametric Framework for Imitation Learning](https://mlsanity.com/paper/1909.07374/) (2019)
- [Generating Large Convex Polytopes Directly on Point Clouds](https://mlsanity.com/paper/2010.08744/) (2020)

## Built on by (most cited first, 30 of 30)

- [HardNet: Hard-Constrained Neural Networks with Universal Approximation Guarantees](https://mlsanity.com/paper/2410.10807/) (2024)
- [CGD: Constraint-Guided Diffusion Policies for UAV Trajectory Planning](https://mlsanity.com/paper/2405.01758/) (2024)
- [FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guarantees](https://mlsanity.com/paper/2506.00362/) (2025)
- [Pinet: Optimizing hard-constrained neural networks with orthogonal projection layers](https://mlsanity.com/paper/2508.10480/) (2025)
- [ENFORCE: Nonlinear Constrained Learning with Adaptive-depth Neural Projection](https://mlsanity.com/paper/2502.06774/) (2025)
- [Dual Lagrangian Learning for Conic Optimization](https://mlsanity.com/paper/2402.03086/) (2024)
- [Enforcing Hard Linear Constraints in Deep Learning Models with Decision Rules](https://mlsanity.com/paper/2505.13858/) (2025)
- [Learning to Optimize for Mixed-Integer Non-linear Programming with Feasibility Guarantees](https://mlsanity.com/paper/2410.11061/) (2024)
- [GLinSAT: The General Linear Satisfiability Neural Network Layer By Accelerated Gradient Descent](https://mlsanity.com/paper/2409.17500/) (2024)
- [LMI-Net: Linear Matrix Inequality--Constrained Neural Networks via Differentiable Projection Layers](https://mlsanity.com/paper/2604.05374/) (2026)
- [Soft-Radial Projection for Constrained End-to-End Learning](https://mlsanity.com/paper/2602.03461/) (2026)
- [Improving Feasibility via Fast Autoencoder-Based Projections](https://mlsanity.com/paper/2604.03489/) (2026)
- [Compact Optimality Verification for Optimization Proxies](https://mlsanity.com/paper/2405.21023/) (2024)
- [Tightening optimality gap with confidence through conformal prediction](https://mlsanity.com/paper/2503.04071/) (2025)
- [Enforcing convex constraints in Graph Neural Networks](https://mlsanity.com/paper/2510.11227/) (2025)
- [ECO: Energy-Constrained Operator Learning for Chaotic Dynamics with Boundedness Guarantees](https://mlsanity.com/paper/2512.01984/) (2025)
- [SnareNet: Flexible Repair Layers for Neural Networks with Hard Constraints](https://mlsanity.com/paper/2602.09317/) (2026)
- [mPOLICE: Provable Enforcement of Multi-Region Affine Constraints in Deep Neural Networks](https://mlsanity.com/paper/2502.02434/) (2025)
- [Cheap Thrills: Effective Amortized Optimization Using Inexpensive Labels](https://mlsanity.com/paper/2603.05495/) (2026)
- [Parametric Nonconvex Optimization via Convex Surrogates](https://mlsanity.com/paper/2604.05640/) (2026)
- [Solving Max-Cut to Global Optimality via Feasibility-Preserving Graph Neural Networks](https://mlsanity.com/paper/2605.07113/) (2026)
- [Learning to Solve Constrained Bilevel Control Co-Design Problems](https://mlsanity.com/paper/2507.09050/) (2025)
- [Self-Certifying Primal-Dual Optimization Proxies for Large-Scale Batch Economic Dispatch](https://mlsanity.com/paper/2510.15850/) (2025)
- [Stochastic Dimension Implicit Functional Projections for Global Integral Conservation in High-Dimensional PINNs](https://mlsanity.com/paper/2603.29237/) (2026)
- [CAffNet: Hard Constraint-Affine Neural Networks](https://mlsanity.com/paper/2605.24437/) (2026)
- [DisjunctiveNet: Neural Symbolic Learning via Differentiable Convexified Optimization Layers](https://mlsanity.com/paper/2605.30456/) (2026)
- [Embedding Linear Equality Constraints in Probabilistic Neural Networks for Dynamic Modelling](https://mlsanity.com/paper/2606.21728/) (2026)
- [CG4AI: A Column Generation Framework for Training AI Models Under Constraints](https://mlsanity.com/paper/2608.26375/) (2026)
- [AT-SKM-Net: An Accelerated Trainable Sampling Kaczmarz-Motzkin Framework for Linear Hard-Constraint Feasibility on Dynamic Graphs](https://mlsanity.com/paper/2609.30088/) (2026)
- [SLP-ProbHard: Probabilistic Hard-Constrained Learning via Structural Latent Parameterization](https://mlsanity.com/paper/2609.33515/) (2026)

## Abstract (arXiv)

Despite the numerous applications of convex constraints in Robotics, enforcing them within learning-based frameworks remains an open challenge. Existing techniques either fail to guarantee satisfaction at all times, or incur prohibitive computational costs. This paper presents RAYEN, a framework for imposing hard convex constraints on the output or latent variables of a neural network. RAYEN guarantees constraint satisfaction during both training and testing, for any input and any network weights. Unlike prior approaches, RAYEN avoids computationally expensive orthogonal projections, soft constraints, conservative approximations of the feasible set, and slow iterative corrections. RAYEN supports any combination of linear, convex quadratic, second-order cone (SOC), and linear matrix inequality (LMI) constraints, with negligible overhead compared to unconstrained networks. For instance, it imposes 1K quadratic constraints on a 1K-dimensional variable with only 8 ms of overhead compared to a network that does not enforce these constraints. An LMI constraint with 300x300 dense matrices on a 10K-dimensional variable can be guaranteed with only 12 ms additional overhead. When used in neural networks that approximate the solution of constrained trajectory optimization problems, RAYEN runs 20 to 7468 times faster than state-of-the-art algorithms, while guaranteeing constraint satisfaction at all times and achieving a near-optimal cost (<1.5% optimality gap). Finally, we demonstrate RAYEN's ability to enforce actuator constraints on a learned locomotion policy by validating constraint satisfaction in both simulation and real-world experiments on a quadruped robot. The code is available at https://github.com/leggedrobotics/rayen