# FORESEE: Prediction with Expansion-Compression Unscented Transform for Online Policy Optimization

arXiv: https://arxiv.org/abs/2209.12644 · HTML: https://mlsanity.com/paper/2209.12644/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 48 references that are in the map.
- First posted on arXiv 26 Sep 2022.

## Builds on (most cited first, 15 of 15)

- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019)
- [Control Barrier Function Based Quadratic Programs for Safety Critical Systems](https://mlsanity.com/paper/1609.06408/) (2016)
- [Uncertainty Quantification and Deep Ensembles](https://mlsanity.com/paper/2007.08792/) (2020)
- [Gaussian Processes for Data-Efficient Learning in Robotics and Control](https://mlsanity.com/paper/1502.02860/) (2015)
- [Cautious Model Predictive Control using Gaussian Process Regression](https://mlsanity.com/paper/1705.10702/) (2017)
- [Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control](https://mlsanity.com/paper/1706.06491/) (2017)
- [safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Robotics](https://mlsanity.com/paper/2109.06325/) (2021)
- [Black-Box Data-efficient Policy Search for Robotics](https://mlsanity.com/paper/1703.07261/) (2017)
- [Model-Based Policy Search Using Monte Carlo Gradient Estimation with Real Systems Application](https://mlsanity.com/paper/2101.12115/) (2021)
- [Recursive Feasibility Guided Optimal Parameter Adaptation of Differential Convex Optimization Policies for Safety-Critical Systems](https://mlsanity.com/paper/2109.10949/) (2021)
- [FISAR: Forward Invariant Safe Reinforcement Learning with a Deep Neural Network-Based Optimize](https://mlsanity.com/paper/2006.11419/) (2020)
- [A Generalized Unscented Transformation for Probability Distributions](https://mlsanity.com/paper/2104.01958/) (2021)
- [Online Constrained Model-based Reinforcement Learning](https://mlsanity.com/paper/2004.03499/) (2020)
- [SafePILCO: a software tool for safe and data-efficient policy synthesis](https://mlsanity.com/paper/2008.03273/) (2020)

## Abstract (arXiv)

Propagating state distributions through a generic, uncertain nonlinear dynamical model is known to be intractable and usually begets numerical or analytical approximations. We introduce a method for state prediction, called the Expansion-Compression Unscented Transform, and use it to solve a class of online policy optimization problems. Our proposed algorithm propagates a finite number of sigma points through a state-dependent distribution, which dictates an increase in the number of sigma points at each time step to represent the resulting distribution; this is what we call the expansion operation. To keep the algorithm scalable, we augment the expansion operation with a compression operation based on moment matching, thereby keeping the number of sigma points constant across predictions over multiple time steps. Its performance is empirically shown to be comparable to Monte Carlo but at a much lower computational cost. Under state and control input constraints, the state prediction is subsequently used in tandem with a proposed variant of constrained gradient-descent for online update of policy parameters in a receding horizon fashion. The framework is implemented as a differentiable computational graph for policy training. We showcase our framework for a quadrotor stabilization task as part of a benchmark comparison in safe-control-gym and for optimizing the parameters of a Control Barrier Function based controller in a leader-follower problem.