# Adaptive World Models: Learning Behaviors by Latent Imagination Under Non-Stationarity

arXiv: https://arxiv.org/abs/2411.01342 · HTML: https://mlsanity.com/paper/2411.01342/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in World models for navigation, one of 11 directions in World models.
- Builds on 22 of its 28 references that are in the map.
- First posted on arXiv 2 Nov 2024.

## Placed in

- World models / [World models for navigation](https://mlsanity.com/wm/world-models-for-navigation/)

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [TD-MPC2: Scalable, Robust World Models for Continuous Control](https://mlsanity.com/paper/2310.16828/) (2023)
- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [dm_control: Software and Tasks for Continuous Control](https://mlsanity.com/paper/2006.12983/) (2020)
- [Towards Continual Reinforcement Learning: A Review and Perspectives](https://mlsanity.com/paper/2012.13490/) (2020)
- [Deep Kalman Filters](https://mlsanity.com/paper/1511.05121/) (2015)
- [An empirical investigation of the challenges of real-world reinforcement learning](https://mlsanity.com/paper/2003.11881/) (2020)
- [Hidden Parameter Markov Decision Processes: A Semiparametric Regression Approach for Discovering Latent Task Parametrizations](https://mlsanity.com/paper/1308.3513/) (2013)
- [Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep Feature Spaces](https://mlsanity.com/paper/1905.07357/) (2019)
- [Contextualize Me -- The Case for Context in Reinforcement Learning](https://mlsanity.com/paper/2202.04500/) (2022)
- [Deep Reinforcement Learning amidst Lifelong Non-Stationarity](https://mlsanity.com/paper/2006.10701/) (2020)
- [Mastering Memory Tasks with World Models](https://mlsanity.com/paper/2403.04253/) (2024)
- [The Essential Role of Causality in Foundation World Models for Embodied AI](https://mlsanity.com/paper/2402.06665/) (2024)
- [Action-Conditional Recurrent Kalman Networks For Forward and Inverse Dynamics Learning](https://mlsanity.com/paper/2010.10201/) (2020)
- [On Uncertainty in Deep State Space Models for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2210.09256/) (2022)
- [Hidden Parameter Recurrent State Space Models For Changing Dynamics Scenarios](https://mlsanity.com/paper/2206.14697/) (2022)
- [Multi Time Scale World Models](https://mlsanity.com/paper/2310.18534/) (2023)
- [Learning World Models With Hierarchical Temporal Abstractions: A Probabilistic Perspective](https://mlsanity.com/paper/2404.16078/) (2024)

## Abstract (arXiv)

Developing foundational world models is a key research direction for embodied intelligence, with the ability to adapt to non-stationary environments being a crucial criterion. In this work, we introduce a new formalism, Hidden Parameter-POMDP, designed for control with adaptive world models. We demonstrate that this approach enables learning robust behaviors across a variety of non-stationary RL benchmarks. Additionally, this formalism effectively learns task abstractions in an unsupervised manner, resulting in structured, task-aware latent spaces.