# Koopman Dreamer: Spectrally Constrained Latent Dynamics for Stable World-Model Imagination

arXiv: https://arxiv.org/abs/2607.19719 · HTML: https://mlsanity.com/paper/2607.19719/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Latent predictive world models, one of 11 directions in World models.
- Builds on 34 of its 51 references that are in the map.
- First posted on arXiv 22 Jul 2026.

## Placed in

- World models / [Latent predictive world models](https://mlsanity.com/wm/latent-predictive-world-models/)

## Builds on (most cited first, 30 of 34)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018)
- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [TD-MPC2: Scalable, Robust World Models for Continuous Control](https://mlsanity.com/paper/2310.16828/) (2023)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Maximum a Posteriori Policy Optimisation](https://mlsanity.com/paper/1806.06920/) (2018)
- [Unitary Evolution Recurrent Neural Networks](https://mlsanity.com/paper/1511.06464/) (2015)
- [Modern Koopman Theory for Dynamical Systems](https://mlsanity.com/paper/2102.12086/) (2021)
- [Distributed Distributional Deterministic Policy Gradients](https://mlsanity.com/paper/1804.08617/) (2018)
- [Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition](https://mlsanity.com/paper/1710.04340/) (2017)
- [Acme: A Research Framework for Distributed Reinforcement Learning](https://mlsanity.com/paper/2006.00979/) (2020)
- [Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control](https://mlsanity.com/paper/1611.03537/) (2016)
- [Stable Recurrent Models](https://mlsanity.com/paper/1805.10369/) (2018)
- [Data-driven discovery of Koopman eigenfunctions for control](https://mlsanity.com/paper/1707.01146/) (2017)
- [A Distributional View on Multi-Objective Policy Optimization](https://mlsanity.com/paper/2005.07513/) (2020)
- [Koopman Q-learning: Offline Reinforcement Learning via Symmetries of Dynamics](https://mlsanity.com/paper/2111.01365/) (2021)
- [Deep Koopman Operator with Control for Nonlinear Systems](https://mlsanity.com/paper/2202.08004/) (2022)
- [Air Learning: A Deep Reinforcement Learning Gym for Autonomous Aerial Robot Visual Navigation](https://mlsanity.com/paper/1906.00421/) (2019)
- [Task-Oriented Koopman-Based Control with Contrastive Encoder](https://mlsanity.com/paper/2309.16077/) (2023)

## Built on by (most cited first, 4 of 4)

- [Rollout-Decoded Reconstruction for Long-Horizon Prediction in Latent World Models](https://mlsanity.com/paper/2608.25017/) (2026)
- [Making Latent Evolution Explicit: Operator-Structured Transitions for World Action Models](https://mlsanity.com/paper/2608.27259/) (2026)
- [DWMP: Leveraging Dual World Models for Humanoid Obstacle Traversal](https://mlsanity.com/paper/2609.12347/) (2026)
- [When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Models](https://mlsanity.com/paper/2610.05550/) (2026)

## Abstract (arXiv)

Latent world models improve sample efficiency in continuous control by optimizing policies over imagined latent trajectories, but common neural transitions offer limited direct control over modal persistence and error accumulation in long rollouts. We propose Koopman Dreamer, a Dreamer-style world model with a spectrally constrained deterministic latent dynamics core. Its Koopman-inspired backbone uses two-dimensional rotation--scaling blocks with bounded radii to represent damping, rotation, and near-periodic modes. Linear and low-rank bilinear action terms capture global and state-dependent control effects, while stochastic-state modulation supplies local correction information. To reduce the mismatch between posterior-conditioned training and prior-only imagination, the model combines posterior-conditioned EMA teacher targets with one-step consistency, multi-step rollout, and open-loop observation-prediction objectives. We further derive a multi-step rollout-error bound that separates amplification by the spectral backbone and bilinear interaction from the additive effects of stochastic-state mismatch and modeling residuals, clarifying the trade-off between error attenuation and long-term information retention. Experimental results on proprioceptive continuous-control tasks from the DeepMind Control Suite and UAV-LiDAR autonomous navigation demonstrate that Koopman Dreamer improves the stability of long-horizon latent rollouts and achieves stronger closed-loop control performance on tasks that rely on high-quality multi-step imagination.