# Compactly Restrictable Metric Policy Optimization Problems

arXiv: https://arxiv.org/abs/2207.05850 · HTML: https://mlsanity.com/paper/2207.05850/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 42 references that are in the map.
- First posted on arXiv 12 Jul 2022.

## Builds on (most cited first, 11 of 11)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019)
- [Control Barrier Function Based Quadratic Programs for Safety Critical Systems](https://mlsanity.com/paper/1609.06408/) (2016)
- [Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions](https://mlsanity.com/paper/2004.07584/) (2020)
- [Learning Model Predictive Control for iterative tasks. A Data-Driven Control Framework](https://mlsanity.com/paper/1609.01387/) (2016)
- [MDPs with Setwise Continuous Transition Probabilities](https://mlsanity.com/paper/2011.01325/) (2020)
- [A Mixed Value and Policy Iteration Method for Stochastic Control with Universally Measurable Policies](https://mlsanity.com/paper/1308.3814/) (2013)
- [Sampled-Data Stabilization with Control Lyapunov Functions via Quadratically Constrained Quadratic Programs](https://mlsanity.com/paper/2103.03937/) (2021)
- [Safety of Sampled-Data Systems with Control Barrier Functions via Approximate Discrete Time Models](https://mlsanity.com/paper/2203.11470/) (2022)

## Abstract (arXiv)

We study policy optimization problems for deterministic Markov decision processes (MDPs) with metric state and action spaces, which we refer to as Metric Policy Optimization Problems (MPOPs). Our goal is to establish theoretical results on the well-posedness of MPOPs that can characterize practically relevant continuous control systems. To do so, we define a special class of MPOPs called Compactly Restrictable MPOPs (CR-MPOPs), which are flexible enough to capture the complex behavior of robotic systems but specific enough to admit solutions using dynamic programming methods such as value iteration. We show how to arrive at CR-MPOPs using forward-invariance. We further show that our theoretical results on CR-MPOPs can be used to characterize feedback linearizable control affine systems.