# Reinforcement Learning-based Control via Y-wise Affine Neural Networks (YANNs)

arXiv: https://arxiv.org/abs/2508.16474 · HTML: https://mlsanity.com/paper/2508.16474/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 24 of its 72 references that are in the map.
- First posted on arXiv 22 Aug 2025.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [A Lyapunov-based Approach to Safe Reinforcement Learning](https://mlsanity.com/paper/1805.07708/) (2018)
- [Safe Exploration in Continuous Action Spaces](https://mlsanity.com/paper/1801.08757/) (2018)
- [Reinforcement learning](https://mlsanity.com/paper/2405.10369/) (2024)
- [Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones](https://mlsanity.com/paper/2010.15920/) (2020)
- [Stochastic Variance-Reduced Policy Gradient](https://mlsanity.com/paper/1806.05618/) (2018)
- [Safe Reinforcement Learning Using Robust MPC](https://mlsanity.com/paper/1906.04005/) (2019)
- [Data-driven Economic NMPC using Reinforcement Learning](https://mlsanity.com/paper/1904.04152/) (2019)
- [Stabilizing Neural Control Using Self-Learned Almost Lyapunov Critics](https://mlsanity.com/paper/2107.04989/) (2021)
- [Reinforcement Learning for Batch Bioprocess Optimization](https://mlsanity.com/paper/1904.07292/) (2019)
- [Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning](https://mlsanity.com/paper/2111.07171/) (2021)
- [Deep Reinforcement Learning for Process Control: A Primer for Beginners](https://mlsanity.com/paper/2004.05490/) (2020)
- [Safe Chance Constrained Reinforcement Learning for Batch Process Control](https://mlsanity.com/paper/2104.11706/) (2021)
- [Control-Informed Reinforcement Learning for Chemical Processes](https://mlsanity.com/paper/2408.13566/) (2024)
- [AC4MPC: Actor-Critic Reinforcement Learning for Nonlinear Model Predictive Control](https://mlsanity.com/paper/2406.03995/) (2024)
- [Twin actor twin delayed deep deterministic policy gradient (TATD3) learning for batch process control](https://mlsanity.com/paper/2102.13012/) (2021)
- [Reinforcement Learning for Control of Valves](https://mlsanity.com/paper/2012.14668/) (2020)
- [A Reinforcement Learning-based Economic Model Predictive Control Framework for Autonomous Operation of Chemical Reactors](https://mlsanity.com/paper/2105.02656/) (2021)
- [Control invariant set enhanced safe reinforcement learning: improved sampling efficiency, guaranteed stability and robustness](https://mlsanity.com/paper/2305.15602/) (2023)
- [YANNs: Y-wise Affine Neural Networks for Exact and Efficient Representations of Piecewise Linear Functions](https://mlsanity.com/paper/2505.07054/) (2025)

## Built on by (most cited first, 2 of 2)

- [Reinforcement Learning-based Control via Y-wise Affine Neural Networks: Comparative Case Studies for Chemical Processes](https://mlsanity.com/paper/2605.21211/) (2026)
- [Enhanced Reinforcement Learning-based Process Synthesis via Quantum Computing](https://mlsanity.com/paper/2605.21213/) (2026)

## Abstract (arXiv)

This work presents a novel reinforcement learning (RL) algorithm based on Y-wise Affine Neural Networks (YANNs). YANNs provide an interpretable neural network which can exactly represent known piecewise affine functions of arbitrary input and output dimensions defined on any amount of polytopic subdomains. One representative application of YANNs is to reformulate explicit solutions of multi-parametric linear model predictive control. Built on this, we propose the use of YANNs to initialize RL actor and critic networks, which enables the resulting YANN-RL control algorithm to start with the confidence of linear optimal control. The YANN-actor is initialized by representing the multi-parametric control solutions obtained via offline computation using an approximated linear system model. The YANN-critic represents the explicit form of the state-action value function for the linear system and the reward function as the objective in an optimal control problem (OCP). Additional network layers are injected to extend YANNs for nonlinear expressions, which can be trained online by directly interacting with the true complex nonlinear system. In this way, both the policy and state-value functions exactly represent a linear OCP initially and are able to eventually learn the solution of a general nonlinear OCP. Continuous policy improvement is also implemented to provide heuristic confidence that the linear OCP solution serves as an effective lower bound to the performance of RL policy. The YANN-RL algorithm is demonstrated on a clipped pendulum and a safety-critical chemical-reactive system. Our results show that YANN-RL significantly outperforms the modern RL algorithm using deep deterministic policy gradient, especially when considering safety constraints.