# Safe Reinforcement Learning of Control-Affine Systems with Vertex Networks

arXiv: https://arxiv.org/abs/2003.09488 · HTML: https://mlsanity.com/paper/2003.09488/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2020.

## Built on by (most cited first, 8 of 8)

- [Gradient Shaping for Multi-Constraint Safe Reinforcement Learning](https://mlsanity.com/paper/2312.15127/) (2023)
- [Enforcing Hard Linear Constraints in Deep Learning Models with Decision Rules](https://mlsanity.com/paper/2505.13858/) (2025)
- [Constrained Meta-Reinforcement Learning for Adaptable Safety Guarantee with Differentiable Convex Programming](https://mlsanity.com/paper/2312.10230/) (2023)
- [Model-Based Safe Reinforcement Learning with Time-Varying State and Control Constraints: An Application to Intelligent Vehicles](https://mlsanity.com/paper/2112.11217/) (2021)
- [On the Optimality, Stability, and Feasibility of Control Barrier Functions: An Adaptive Learning-Based Approach](https://mlsanity.com/paper/2305.03608/) (2023)
- [Multi-market Energy Optimization with Renewables via Reinforcement Learning](https://mlsanity.com/paper/2306.08147/) (2023)
- [Distributionally Safe Reinforcement Learning under Model Uncertainty: A Single-Level Approach by Differentiable Convex Programming](https://mlsanity.com/paper/2310.02459/) (2023)
- [DisjunctiveNet: Neural Symbolic Learning via Differentiable Convexified Optimization Layers](https://mlsanity.com/paper/2605.30456/) (2026)

## Abstract (arXiv)

This paper focuses on finding reinforcement learning policies for control systems with hard state and action constraints. Despite its success in many domains, reinforcement learning is challenging to apply to problems with hard constraints, especially if both the state variables and actions are constrained. Previous works seeking to ensure constraint satisfaction, or safety, have focused on adding a projection step to a learned policy. Yet, this approach requires solving an optimization problem at every policy execution step, which can lead to significant computational costs. To tackle this problem, this paper proposes a new approach, termed Vertex Networks (VNs), with guarantees on safety during exploration and on learned control policies by incorporating the safety constraints into the policy network architecture. Leveraging the geometric property that all points within a convex set can be represented as the convex combination of its vertices, the proposed algorithm first learns the convex combination weights and then uses these weights along with the pre-calculated vertices to output an action. The output action is guaranteed to be safe by construction. Numerical examples illustrate that the proposed VN algorithm outperforms vanilla reinforcement learning in a variety of benchmark control tasks.