mlsanityData to 7 Oct 2026

FORESEE: Prediction with Expansion-Compression Unscented Transform for Online Policy Optimization

arXiv 2209.12644 ↗First posted 26 Sep 2022cs.RO cs.LG cs.SYcited by 0

Not on the map. Why?

Abstract (arXiv)

Propagating state distributions through a generic, uncertain nonlinear dynamical model is known to be intractable and usually begets numerical or analytical approximations. We introduce a method for state prediction, called the Expansion-Compression Unscented Transform, and use it to solve a class of online policy optimization problems. Our proposed algorithm propagates a finite number of sigma points through a state-dependent distribution, which dictates an increase in the number of sigma points at each time step to represent the resulting distribution; this is what we call the expansion operation. To keep the algorithm scalable, we augment the expansion operation with a compression operation based on moment matching, thereby keeping the number of sigma points constant across predictions over multiple time steps. Its performance is empirically shown to be comparable to Monte Carlo but at a much lower computational cost. Under state and control input constraints, the state prediction is subsequently used in tandem with a proposed variant of constrained gradient-descent for online update of policy parameters in a receding horizon fashion. The framework is implemented as a differentiable computational graph for policy training. We showcase our framework for a quadrotor stabilization task as part of a benchmark comparison in safe-control-gym and for optimizing the parameters of a Control Barrier Function based controller in a leader-follower problem.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 48 references that are in the map. First posted on arXiv 26 Sep 2022.

Builds on

15 of 48 references.

References · 15

  1. 2019Deep Evidential Regressioncited by 354
  2. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 261
  3. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
  4. 2020Uncertainty Quantification and Deep Ensemblescited by 138
  5. 2015Gaussian Processes for Data-Efficient Learning in Robotics and Controlcited by 97
  6. 2017Cautious Model Predictive Control using Gaussian Process Regressioncited by 91
  7. 2017Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Controlcited by 50
  8. 2021safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Roboticscited by 42
  9. 2017Black-Box Data-efficient Policy Search for Roboticscited by 14
  10. 2021Model-Based Policy Search Using Monte Carlo Gradient Estimation with Real Systems Applicationcited by 7
  11. 2021Recursive Feasibility Guided Optimal Parameter Adaptation of Differential Convex Optimization Policies for Safety-Critical Systemscited by 7
  12. 2020FISAR: Forward Invariant Safe Reinforcement Learning with a Deep Neural Network-Based Optimizecited by 5
3 more
  1. 2021A Generalized Unscented Transformation for Probability Distributionscited by 4
  2. 2020Online Constrained Model-based Reinforcement Learningcited by 3
  3. 2020SafePILCO: a software tool for safe and data-efficient policy synthesiscited by 3

Built on by

Not cited yet.