# MAGIC: Learning Macro-Actions for Online POMDP Planning

arXiv: https://arxiv.org/abs/2011.03813 · HTML: https://mlsanity.com/paper/2011.03813/

## Key facts

- Cited by 17 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 7 Nov 2020.

## Built on by (most cited first, 17 of 17)

- [Partially Observable Markov Decision Processes in Robotics: A Survey](https://mlsanity.com/paper/2209.10342/) (2022)
- [Asynchronous Actor-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2209.10113/) (2022)
- [Towards Optimal Correlational Object Search](https://mlsanity.com/paper/2110.09991/) (2021)
- [Closing the Planning-Learning Loop with Application to Autonomous Driving](https://mlsanity.com/paper/2101.03834/) (2021)
- [LEADER: Learning Attention over Driving Behaviors for Planning under Uncertainty](https://mlsanity.com/paper/2209.11422/) (2022)
- [Intention-Aware Navigation in Crowds with Extended-Space POMDP Planning](https://mlsanity.com/paper/2206.10028/) (2022)
- [Smoother Entropy for Active State Trajectory Estimation and Obfuscation in POMDPs](https://mlsanity.com/paper/2108.10227/) (2021)
- [Learning Symbolic Persistent Macro-Actions for POMDP Solving Over Time](https://mlsanity.com/paper/2505.03668/) (2025)
- [RiskMap: A Unified Driving Context Representation for Autonomous Motion Planning in Urban Driving Environment](https://mlsanity.com/paper/2406.04451/) (2024)
- [Hi-Drive: Hierarchical POMDP Planning for Safe Autonomous Driving in Diverse Urban Environments](https://mlsanity.com/paper/2409.18411/) (2024)
- [The Planner Optimization Problem: Formulations and Frameworks](https://mlsanity.com/paper/2303.06768/) (2023)
- [PlanScope: Learning to Plan Within Decision Scope for Urban Autonomous Driving](https://mlsanity.com/paper/2411.00476/) (2024)
- [A Convolution and Attention Based Encoder for Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/2505.23857/) (2025)
- [Meta-learning how to Share Credit among Macro-Actions](https://mlsanity.com/paper/2506.13690/) (2025)
- [Partially Observable Reference Policy Programming: Solving POMDPs Sans Numerical Optimisation](https://mlsanity.com/paper/2507.12186/) (2025)
- [Vec-QMDP: Vectorized POMDP Planning on CPUs for Real-Time Autonomous Driving](https://mlsanity.com/paper/2602.08334/) (2026)
- [VOiLA: Vectorized Online Planning with Learned Diffusion Models for POMDP Agents](https://mlsanity.com/paper/2606.19729/) (2026)

## Abstract (arXiv)

The partially observable Markov decision process (POMDP) is a principled general framework for robot decision making under uncertainty, but POMDP planning suffers from high computational complexity, when long-term planning is required. While temporally-extended macro-actions help to cut down the effective planning horizon and significantly improve computational efficiency, how do we acquire good macro-actions? This paper proposes Macro-Action Generator-Critic (MAGIC), which performs offline learning of macro-actions optimized for online POMDP planning. Specifically, MAGIC learns a macro-action generator end-to-end, using an online planner's performance as the feedback. During online planning, the generator generates on the fly situation-aware macro-actions conditioned on the robot's belief and the environment context. We evaluated MAGIC on several long-horizon planning tasks both in simulation and on a real robot. The experimental results show that the learned macro-actions offer significant benefits in online planning performance, compared with primitive actions and handcrafted macro-actions.