# Discover Life Skills for Planning with Bandits via Observing and Learning How the World Works

arXiv: https://arxiv.org/abs/2207.08130 · HTML: https://mlsanity.com/paper/2207.08130/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 41 references that are in the map.
- First posted on arXiv 17 Jul 2022.

## Builds on (most cited first, 8 of 8)

- [Simple regret for infinitely many armed bandits](https://mlsanity.com/paper/1505.04627/) (2015)
- [The Unreasonable Effectiveness of Greedy Algorithms in Multi-Armed Bandit with Many Arms](https://mlsanity.com/paper/2002.10121/) (2020)
- [Parallelised Diffeomorphic Sampling-based Motion Planning](https://mlsanity.com/paper/2108.11775/) (2021)
- [Balancing Global Exploration and Local-connectivity Exploitation with Rapidly-exploring Random disjointed-Trees](https://mlsanity.com/paper/1810.03749/) (2018)
- [Bayesian Local Sampling-based Planning](https://mlsanity.com/paper/1909.03452/) (2019)
- [Learning to Plan Hierarchically from Curriculum](https://mlsanity.com/paper/1906.07371/) (2019)
- [L4KDE: Learning for KinoDynamic Tree Expansion](https://mlsanity.com/paper/2203.00975/) (2022)
- [Robust Hierarchical Planning with Policy Delegation](https://mlsanity.com/paper/2010.13033/) (2020)

## Built on by (most cited first, 1 of 1)

- [Psy-LLM: Scaling up Global Mental Health Psychological Services with AI-based Large Language Models](https://mlsanity.com/paper/2307.11991/) (2023)

## Abstract (arXiv)

We propose a novel approach for planning agents to compose abstract skills via observing and learning from historical interactions with the world. Our framework operates in a Markov state-space model via a set of actions under unknown pre-conditions. We formulate skills as high-level abstract policies that propose action plans based on the current state. Each policy learns new plans by observing the states' transitions while the agent interacts with the world. Such an approach automatically learns new plans to achieve specific intended effects, but the success of such plans is often dependent on the states in which they are applicable. Therefore, we formulate the evaluation of such plans as infinitely many multi-armed bandit problems, where we balance the allocation of resources on evaluating the success probability of existing arms and exploring new options. The result is a planner capable of automatically learning robust high-level skills under a noisy environment; such skills implicitly learn the action pre-condition without explicit knowledge. We show that this planning approach is experimentally very competitive in high-dimensional state space domains.