# Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping

arXiv: https://arxiv.org/abs/2607.23930 · HTML: https://mlsanity.com/paper/2607.23930/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 25 of its 50 references that are in the map.
- First posted on arXiv 27 Jul 2026.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 25 of 25)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning](https://mlsanity.com/paper/2410.21845/) (2024)
- [ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robots](https://mlsanity.com/paper/2306.14874/) (2023)
- [Stop Regressing: Training Value Functions via Classification for Scalable Deep RL](https://mlsanity.com/paper/2403.03950/) (2024)
- [Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control](https://mlsanity.com/paper/1811.01848/) (2018)
- [Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous control](https://mlsanity.com/paper/2405.16158/) (2024)
- [1000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilities](https://mlsanity.com/paper/2503.14858/) (2025)
- [Hyperspherical Normalization for Scalable Deep Reinforcement Learning](https://mlsanity.com/paper/2502.15280/) (2025)
- [Data-Driven MPC for Quadrotors](https://mlsanity.com/paper/2102.05773/) (2021)
- [Safe Reinforcement Learning via Projection on a Safe Set: How to Achieve Optimality?](https://mlsanity.com/paper/2004.00915/) (2020)
- [Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians](https://mlsanity.com/paper/2102.13073/) (2021)
- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [Safe Reinforcement Learning with Chance-constrained Model Predictive Control](https://mlsanity.com/paper/2112.13941/) (2021)
- [Learning When to Drive in Intersections by Combining Reinforcement Learning and Model Predictive Control](https://mlsanity.com/paper/1908.00177/) (2019)
- [Inverse Optimization via Learning Feasible Regions](https://mlsanity.com/paper/2505.15025/) (2025)
- [A Hierarchical Approach for Strategic Motion Planning in Autonomous Racing](https://mlsanity.com/paper/2212.01607/) (2022)
- [A Safe Reinforcement Learning driven Weights-varying Model Predictive Control for Autonomous Vehicle Motion Control](https://mlsanity.com/paper/2402.02624/) (2024)
- [Adaptive Stochastic Nonlinear Model Predictive Control with Look-ahead Deep Reinforcement Learning for Autonomous Vehicle Motion Control](https://mlsanity.com/paper/2311.04303/) (2023)
- [Learning Robot Trajectories subject to Kinematic Joint Constraints](https://mlsanity.com/paper/2011.00563/) (2020)
- [Intrinsic Benefits of Categorical Distributional Loss: Uncertainty-aware Regularized Exploration in Reinforcement Learning](https://mlsanity.com/paper/2110.03155/) (2021)
- [Action Mapping for Reinforcement Learning in Continuous Environments with Constraints](https://mlsanity.com/paper/2412.04327/) (2024)
- [A Dual Active-Set Solver for Embedded Quadratic Programming Using Recursive LDL' Updates](https://mlsanity.com/paper/2103.16236/) (2021)
- [Optimal Transportation by Orthogonal Coupling Dynamics](https://mlsanity.com/paper/2410.08060/) (2024)

## Abstract (arXiv)

Operating constrained dynamical systems requires controllers to efficiently solve complex tasks while enforcing recursive feasibility and physical constraints. To address these competing requirements, we present Feasible Action for Optimal Control (FAOC), a novel control framework integrating Reinforcement Learning (RL) and Optimal Control (OC). The core contribution is a computationally efficient, optimization-based mapping algorithm that transforms the RL agent's action from a static abstract set into a state-dependent feasible parameter set of the Optimal Control Problem (OCP), guaranteeing instantaneous parameter feasibility. When paired with invariant terminal sets, FAOC guarantees strict recursive feasibility and safe operation, effectively combining the predictable safety of OC with the behavioral flexibility of RL. Unlike prior work, the abstract action space does not require expert tuning, nor is the OCP formulation compromised by the inability of RL to guarantee feasibility. We evaluate FAOC on real-time motion planning for robot table tennis, where simulated experiments demonstrate superior sample efficiency and closed-loop performance compared to state-of-the-art baselines. We open-source the used implementation of the mapping algorithm and OCP for motion planning https://github.com/SonyResearch/feasible_action_for_optimal_control.