# Goal-Conditioned Terminal Value Estimation for Real-time and Multi-task Model Predictive Control

arXiv: https://arxiv.org/abs/2410.04929 · HTML: https://mlsanity.com/paper/2410.04929/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 44 references that are in the map.
- First posted on arXiv 7 Oct 2024.

## Builds on (most cited first, 20 of 20)

- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Automatic Goal Generation for Reinforcement Learning Agents](https://mlsanity.com/paper/1705.06366/) (2017)
- [Visual Reinforcement Learning with Imagined Goals](https://mlsanity.com/paper/1807.04742/) (2018)
- [Goal-Conditioned Reinforcement Learning with Imagined Subgoals](https://mlsanity.com/paper/2107.00541/) (2021)
- [Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control](https://mlsanity.com/paper/1811.01848/) (2018)
- [Unsupervised Control Through Non-Parametric Discriminative Rewards](https://mlsanity.com/paper/1811.11359/) (2018)
- [Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving](https://mlsanity.com/paper/1707.02342/) (2017)
- [Contextual Imagined Goals for Self-Supervised Robotic Learning](https://mlsanity.com/paper/1910.11670/) (2019)
- [A Unified MPC Framework for Whole-Body Dynamic Locomotion and Manipulation](https://mlsanity.com/paper/2103.00946/) (2021)
- [Blending MPC & Value Function Approximation for Efficient Reinforcement Learning](https://mlsanity.com/paper/2012.05909/) (2020)
- [Information Theoretic Model Predictive Q-Learning](https://mlsanity.com/paper/2001.02153/) (2019)
- [Weakly-Supervised Reinforcement Learning for Controllable Behavior](https://mlsanity.com/paper/2004.02860/) (2020)
- [MapGo: Model-Assisted Policy Optimization for Goal-Oriented Tasks](https://mlsanity.com/paper/2105.06350/) (2021)
- [An Efficient Optimal Planning and Control Framework For Quadrupedal Locomotion](https://mlsanity.com/paper/1609.09861/) (2016)
- [Hindsight Expectation Maximization for Goal-conditioned Reinforcement Learning](https://mlsanity.com/paper/2006.07549/) (2020)
- [Online Non-linear Centroidal MPC for Humanoid Robot Locomotion with Step Adjustment](https://mlsanity.com/paper/2203.04489/) (2022)
- [DIABLO: A 6-DoF Wheeled Bipedal Robot Composed Entirely of Direct-Drive Joints](https://mlsanity.com/paper/2407.21500/) (2024)
- [Whole-body model predictive control with rigid contacts via online switching time optimization](https://mlsanity.com/paper/2203.00997/) (2022)

## Abstract (arXiv)

While MPC enables nonlinear feedback control by solving an optimal control problem at each timestep, the computational burden tends to be significantly large, making it difficult to optimize a policy within the control period. To address this issue, one possible approach is to utilize terminal value learning to reduce computational costs. However, the learned value cannot be used for other tasks in situations where the task dynamically changes in the original MPC setup. In this study, we develop an MPC framework with goal-conditioned terminal value learning to achieve multitask policy optimization while reducing computational time. Furthermore, by using a hierarchical control structure that allows the upper-level trajectory planner to output appropriate goal-conditioned trajectories, we demonstrate that a robot model is able to generate diverse motions. We evaluate the proposed method on a bipedal inverted pendulum robot model and confirm that combining goal-conditioned terminal value learning with an upper-level trajectory planner enables real-time control; thus, the robot successfully tracks a target trajectory on sloped terrain.