# Mission-Aligned Learning-Informed Control of Autonomous Systems: Formulation and Foundations

arXiv: https://arxiv.org/abs/2507.04356 · HTML: https://mlsanity.com/paper/2507.04356/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 84 references that are in the map.
- First posted on arXiv 6 Jul 2025.

## Builds on (most cited first, 30 of 35)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05143/) (2016)
- [Learning Explanatory Rules from Noisy Data](https://mlsanity.com/paper/1711.04574/) (2017)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [Assessing Generalization in Deep Reinforcement Learning](https://mlsanity.com/paper/1810.12282/) (2018)
- [Safe Reinforcement Learning Using Robust MPC](https://mlsanity.com/paper/1906.04005/) (2019)
- [Data-driven Economic NMPC using Reinforcement Learning](https://mlsanity.com/paper/1904.04152/) (2019)
- [A Single Time-Scale Stochastic Approximation Method for Nested Stochastic Optimization](https://mlsanity.com/paper/1812.01094/) (2018)
- [Neural Logic Reinforcement Learning](https://mlsanity.com/paper/1904.10729/) (2019)
- [Control Regularization for Reduced Variance Reinforcement Learning](https://mlsanity.com/paper/1905.05380/) (2019)
- [A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning](https://mlsanity.com/paper/2109.14756/) (2021)
- [Practical Reinforcement Learning of Stabilizing Economic MPC](https://mlsanity.com/paper/1904.04614/) (2019)
- [A Single-Timescale Analysis For Stochastic Approximation With Multiple Coupled Sequences](https://mlsanity.com/paper/2206.10414/) (2022)
- [Learning for MPC with Stability & Safety Guarantees](https://mlsanity.com/paper/2012.07369/) (2020)
- [Online Planner Selection with Graph Neural Networks and Adaptive Scheduling](https://mlsanity.com/paper/1811.00210/) (2018)
- [Reinforcement Learning for Mixed-Integer Problems Based on MPC](https://mlsanity.com/paper/2004.01430/) (2020)
- [Reinforcement Learning based on Scenario-tree MPC for ASVs](https://mlsanity.com/paper/2103.11949/) (2021)
- [MPC-based Reinforcement Learning for Economic Problems with Application to Battery Storage](https://mlsanity.com/paper/2104.02411/) (2021)
- [Inter-Level Cooperation in Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1912.02368/) (2019)
- [MPC-based Reinforcement Learning for a Simplified Freight Mission of Autonomous Surface Vehicles](https://mlsanity.com/paper/2106.08634/) (2021)
- [Lie Group Forced Variational Integrator Networks for Learning and Control of Robot Systems](https://mlsanity.com/paper/2211.16006/) (2022)
- [Trilevel and Multilevel Optimization using Monotone Operator Theory](https://mlsanity.com/paper/2105.09407/) (2021)
- [Reinforcement Learning with Adaptive Regularization for Safe Control of Critical Systems](https://mlsanity.com/paper/2404.15199/) (2024)
- [A descent algorithm for the optimal control of ReLU neural network informed PDEs based on approximate directional derivatives](https://mlsanity.com/paper/2210.07900/) (2022)
- [Learning-based MPC from Big Data Using Reinforcement Learning](https://mlsanity.com/paper/2301.01667/) (2023)

## Abstract (arXiv)

Research, innovation and practical capital investment have been increasing rapidly toward the realization of autonomous physical agents. This includes industrial and service robots, unmanned aerial vehicles, embedded control devices, and a number of other realizations of cybernetic/mechatronic implementations of intelligent autonomous devices. In this paper, we consider a stylized version of robotic care, which would normally involve a two-level Reinforcement Learning procedure that trains a policy for both lower level physical movement decisions as well as higher level conceptual tasks and their sub-components. In order to deliver greater safety and reliability in the system, we present the general formulation of this as a two-level optimization scheme which incorporates control at the lower level, and classical planning at the higher level, integrated with a capacity for learning. This synergistic integration of multiple methodologies -- control, classical planning, and RL -- presents an opportunity for greater insight for algorithm development, leading to more efficient and reliable performance. Here, the notion of reliability pertains to physical safety and interpretability into an otherwise black box operation of autonomous agents, concerning users and regulators. This work presents the necessary background and general formulation of the optimization framework, detailing each component and its integration with the others.