# Hierarchical Policy Blending as Inference for Reactive Robot Control

arXiv: https://arxiv.org/abs/2210.07890 · HTML: https://mlsanity.com/paper/2210.07890/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 65 references that are in the map.
- First posted on arXiv 14 Oct 2022.

## Builds on (most cited first, 28 of 28)

- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [Sample-efficient Cross-Entropy Method for Real-time Planning](https://mlsanity.com/paper/2008.06389/) (2020)
- [Distributed Gaussian Processes](https://mlsanity.com/paper/1502.02843/) (2015)
- [Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions](https://mlsanity.com/paper/1410.7827/) (2014)
- [Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictors](https://mlsanity.com/paper/2006.13205/) (2020)
- [Neural Dynamic Policies for End-to-End Sensorimotor Learning](https://mlsanity.com/paper/2012.02788/) (2020)
- [Continuous-Time Gaussian Process Motion Planning via Probabilistic Inference](https://mlsanity.com/paper/1707.07383/) (2017)
- [STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulation](https://mlsanity.com/paper/2104.13542/) (2021)
- [Riemannian Motion Policies](https://mlsanity.com/paper/1801.02854/) (2018)
- [Generalized Robust Bayesian Committee Machine for Large-scale Gaussian Process Regression](https://mlsanity.com/paper/1806.00720/) (2018)
- [Stein Variational Model Predictive Control](https://mlsanity.com/paper/2011.07641/) (2020)
- [Specializing Versatile Skill Libraries using Local Mixture of Experts](https://mlsanity.com/paper/2112.04216/) (2021)
- [Robot Learning of Mobile Manipulation with Reachability Behavior Priors](https://mlsanity.com/paper/2203.04051/) (2022)
- [RMPflow: A Computational Graph for Automatic Motion Policy Generation](https://mlsanity.com/paper/1811.07049/) (2018)
- [Learning Implicit Priors for Motion Optimization](https://mlsanity.com/paper/2204.05369/) (2022)
- [Geometric Fabrics: Generalizing Classical Mechanics to Capture the Physics of Behavior](https://mlsanity.com/paper/2109.10443/) (2021)
- [Continuous Action Reinforcement Learning from a Mixture of Interpretable Experts](https://mlsanity.com/paper/2006.05911/) (2020)
- [Hierarchical Neural Dynamic Policies](https://mlsanity.com/paper/2107.05627/) (2021)
- [Composable Energy Policies for Reactive Motion Generation and Reinforcement Learning](https://mlsanity.com/paper/2105.04962/) (2021)
- [Merging Position and Orientation Motion Primitives](https://mlsanity.com/paper/2003.11507/) (2020)
- [Generalized Nonlinear and Finsler Geometry for Robotics](https://mlsanity.com/paper/2010.14745/) (2020)
- [RMP2: A Structured Composable Policy Class for Robot Learning](https://mlsanity.com/paper/2103.05922/) (2021)
- [Geometric Fabrics for the Acceleration-based Design of Robotic Motion](https://mlsanity.com/paper/2010.14750/) (2020)
- [Riemannian Motion Policy Fusion through Learnable Lyapunov Function Reshaping](https://mlsanity.com/paper/1910.02646/) (2019)
- [Learning from demonstration using products of experts: applications to manipulation and task prioritization](https://mlsanity.com/paper/2010.03505/) (2020)
- [Learning to Sequence and Blend Robot Skills via Differentiable Optimization](https://mlsanity.com/paper/2206.00559/) (2022)
- [Transductive Log Opinion Pool of Gaussian Process Experts](https://mlsanity.com/paper/1511.07551/) (2015)
- [Actor-Critic based Improper Reinforcement Learning](https://mlsanity.com/paper/2207.09090/) (2022)

## Built on by (most cited first, 8 of 8)

- [Efficient Diffusion Transformer Policies with Mixture of Expert Denoisers for Multitask Learning](https://mlsanity.com/paper/2412.12953/) (2024)
- [PoCo: Policy Composition from and for Heterogeneous Robot Learning](https://mlsanity.com/paper/2402.02511/) (2024)
- [Variational Distillation of Diffusion Policies into Mixture of Experts](https://mlsanity.com/paper/2406.12538/) (2024)
- [Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Experts](https://mlsanity.com/paper/2403.06966/) (2024)
- [Hierarchical Policy Blending As Optimal Transport](https://mlsanity.com/paper/2212.01938/) (2022)
- [MoVEInt: Mixture of Variational Experts for Learning Human-Robot Interactions from Demonstrations](https://mlsanity.com/paper/2407.07636/) (2024)
- [A Comprehensive Survey on Physical Risk Control in the Era of Foundation Model-enabled Robotics](https://mlsanity.com/paper/2505.12583/) (2025)
- [Pseudo-Likelihood Inference](https://mlsanity.com/paper/2311.16656/) (2023)

## Abstract (arXiv)

Motion generation in cluttered, dense, and dynamic environments is a central topic in robotics, rendered as a multi-objective decision-making problem. Current approaches trade-off between safety and performance. On the one hand, reactive policies guarantee fast response to environmental changes at the risk of suboptimal behavior. On the other hand, planning-based motion generation provides feasible trajectories, but the high computational cost may limit the control frequency and thus safety. To combine the benefits of reactive policies and planning, we propose a hierarchical motion generation method. Moreover, we adopt probabilistic inference methods to formalize the hierarchical model and stochastic optimization. We realize this approach as a weighted product of stochastic, reactive expert policies, where planning is used to adaptively compute the optimal weights over the task horizon. This stochastic optimization avoids local optima and proposes feasible reactive plans that find paths in cluttered and dense environments. Our extensive experimental study in planar navigation and 6DoF manipulation shows that our proposed hierarchical motion generation method outperforms both myopic reactive controllers and online re-planning methods.