# Epistemic Exploration for Generalizable Planning and Learning in Non-Stationary Settings

arXiv: https://arxiv.org/abs/2402.08145 · HTML: https://mlsanity.com/paper/2402.08145/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 26 references that are in the map.
- First posted on arXiv 13 Feb 2024.

## Builds on (most cited first, 6 of 6)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [RAPid-Learn: A Framework for Learning to Recover for Handling Novelties in Open-World Environments](https://mlsanity.com/paper/2206.12493/) (2022)
- [Relational Abstractions for Generalized Reinforcement Learning on Symbolic Problems](https://mlsanity.com/paper/2204.12665/) (2022)
- [Autonomous Capability Assessment of Sequential Decision-Making Systems in Stochastic Settings (Extended Version)](https://mlsanity.com/paper/2306.04806/) (2023)
- [A Domain-Independent Agent Architecture for Adaptive Operation in Evolving Open Worlds](https://mlsanity.com/paper/2306.06272/) (2023)

## Built on by (most cited first, 1 of 1)

- [Autonomous Option Invention for Continual Hierarchical Reinforcement Learning and Planning](https://mlsanity.com/paper/2412.16395/) (2024)

## Abstract (arXiv)

This paper introduces a new approach for continual planning and model learning in relational, non-stationary stochastic environments. Such capabilities are essential for the deployment of sequential decision-making systems in the uncertain and constantly evolving real world. Working in such practical settings with unknown (and non-stationary) transition systems and changing tasks, the proposed framework models gaps in the agent's current state of knowledge and uses them to conduct focused, investigative explorations. Data collected using these explorations is used for learning generalizable probabilistic models for solving the current task despite continual changes in the environment dynamics. Empirical evaluations on several non-stationary benchmark domains show that this approach significantly outperforms planning and RL baselines in terms of sample complexity. Theoretical results show that the system exhibits desirable convergence properties when stationarity holds.