# HOLO-MPPI: Multi-Scenario Motion Planning via Hierarchical Policy Optimization

arXiv: https://arxiv.org/abs/2606.16480 · HTML: https://mlsanity.com/paper/2606.16480/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 15 Jun 2026.

## Builds on (most cited first, 18 of 18)

- [Data-Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1805.08296/) (2018)
- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [Quantifying Generalization in Reinforcement Learning](https://mlsanity.com/paper/1812.02341/) (2018)
- [FeUdal Networks for Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1703.01161/) (2017)
- [Inverse Reward Design](https://mlsanity.com/paper/1711.02827/) (2017)
- [Sample-efficient Cross-Entropy Method for Real-time Planning](https://mlsanity.com/paper/2008.06389/) (2020)
- [Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving](https://mlsanity.com/paper/1707.02342/) (2017)
- [DTC: Deep Tracking Control](https://mlsanity.com/paper/2309.15462/) (2023)
- [Social Attention for Autonomous Decision-Making in Dense Traffic](https://mlsanity.com/paper/1911.12250/) (2019)
- [Policy Search for Model Predictive Control with Application to Agile Drone Flight](https://mlsanity.com/paper/2112.03850/) (2021)
- [Transformer-based Model Predictive Control: Trajectory Optimization via Sequence Modeling](https://mlsanity.com/paper/2410.23916/) (2024)
- [CAJun: Continuous Adaptive Jumping using a Learned Centroidal Controller](https://mlsanity.com/paper/2306.09557/) (2023)
- [Trajectory Distribution Control for Model Predictive Path Integral Control using Covariance Steering](https://mlsanity.com/paper/2109.12147/) (2021)
- [RAMBO: RL-Augmented Model-Based Whole-Body Control for Loco-Manipulation](https://mlsanity.com/paper/2504.06662/) (2025)
- [Towards Efficient MPPI Trajectory Generation with Unscented Guidance: U-MPPI Control Strategy](https://mlsanity.com/paper/2306.12369/) (2023)
- [Hierarchical Vision-Language Planning for Multi-Step Humanoid Manipulation](https://mlsanity.com/paper/2506.22827/) (2025)
- [Sumo: Dynamic and Generalizable Whole-Body Loco-Manipulation](https://mlsanity.com/paper/2604.08508/) (2026)
- [IANN-MPPI: Interaction-Aware Neural Network-Enhanced Model Predictive Path Integral Approach for Autonomous Driving](https://mlsanity.com/paper/2507.11940/) (2025)

## Abstract (arXiv)

Robots deployed in the real world must plan motions across diverse scenarios without per-scenario retuning. End-to-end reinforcement learning (RL) can generalize across scenarios but often becomes brittle under distribution shift, reward misspecification, and stochastic interactions. Model predictive path integral (MPPI) control enables strong real-time refinement without gradients, but its performance depends on a well-shaped sampling prior, while manually designing the priors does not scale to multi-scenario deployment. We present HOLO-MPPI (High-level Offline, Low-level Online MPPI), a multi-scenario motion planning framework that combines high-level policy learning with low-level stochastic optimal control. Offline, we learn a high-level policy that proposes scenario-robust plans in an abstract action space, with a learned world model for online rollout. Online, the policy serves as a data-driven prior generator that parameterizes MPPI's sampling distribution conditioned on the current observation and goal. MPPI then optimizes low-level control sequences around this prior in real time to adapt to local disturbances. We instantiate HOLO-MPPI in autonomous driving by designing an effective high-level action space and tailored model architectures. Our evaluation across diverse driving scenarios shows that HOLO-MPPI improves upon MPPI and end-to-end RL baselines while maintaining real-time control.