# Adapting World Models with Latent-State Dynamics Residuals

arXiv: https://arxiv.org/abs/2504.02252 · HTML: https://mlsanity.com/paper/2504.02252/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Latent dynamics models and model-based RL, one of 11 directions in World models.
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 3 Apr 2025.

## Placed in

- World models / [Latent dynamics models and model-based RL](https://mlsanity.com/wm/latent-dynamics-models-and-model-based-rl/)
- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 23 of 23)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [TD-MPC2: Scalable, Robust World Models for Continuous Control](https://mlsanity.com/paper/2310.16828/) (2023)
- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [Planning to Explore via Self-Supervised World Models](https://mlsanity.com/paper/2005.05960/) (2020)
- [Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks](https://mlsanity.com/paper/1812.07252/) (2018)
- [Active Domain Randomization](https://mlsanity.com/paper/1904.04762/) (2019)
- [Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environment](https://mlsanity.com/paper/2104.05632/) (2021)
- [TuneNet: One-Shot Residual Tuning for System Identification and Sim-to-Real Robot Task Transfer](https://mlsanity.com/paper/1907.11200/) (2019)
- [Bayesian Multi-Task Learning MPC for Robotic Mobile Manipulation](https://mlsanity.com/paper/2211.10270/) (2022)
- [Grasp Stability Prediction with Sim-to-Real Transfer from Tactile Sensing](https://mlsanity.com/paper/2208.02885/) (2022)
- [Real-to-Sim: Predicting Residual Errors of Robotic Systems with Sparse Data using a Learning-based Unscented Kalman Filter](https://mlsanity.com/paper/2209.03210/) (2022)
- [Make the Pertinent Salient: Task-Relevant Reconstruction for Visual Control with Distractions](https://mlsanity.com/paper/2410.09972/) (2024)
- [Affine Transport for Sim-to-Real Domain Adaptation](https://mlsanity.com/paper/2105.11739/) (2021)

## Built on by (most cited first, 9 of 9)

- [A Comprehensive Survey on World Models for Embodied AI](https://mlsanity.com/paper/2510.16732/) (2025)
- [AdaJEPA: An Adaptive Latent World Model](https://mlsanity.com/paper/2606.32026/) (2026)
- [Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation](https://mlsanity.com/paper/2603.15759/) (2026)
- [World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications](https://mlsanity.com/paper/2606.00133/) (2026)
- [WorldAgen: Unified State-Action Prediction with Test-Time World Model Training](https://mlsanity.com/paper/2609.08162/) (2026)
- [Amortized Low-Rank Adaptation for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2609.12278/) (2026)
- [DeepJEPA: Scaling World Models from Within](https://mlsanity.com/paper/2610.00368/) (2026)
- [Calibration-risk routing for controlled world-model adaptation](https://mlsanity.com/paper/2610.01001/) (2026)
- [SimForcing: Distilling Simulation Motion Priors into Real-Domain Robot World Models](https://mlsanity.com/paper/2610.06598/) (2026)

## Abstract (arXiv)

Simulation-to-reality reinforcement learning (RL) faces the critical challenge of reconciling discrepancies between simulated and real-world dynamics, which can severely degrade agent performance. A promising approach involves learning corrections to simulator forward dynamics represented as a residual error function, however this operation is impractical with high-dimensional states such as images. To overcome this, we propose ReDRAW, a latent-state autoregressive world model pretrained in simulation and calibrated to target environments through residual corrections of latent-state dynamics rather than of explicit observed states. Using this adapted world model, ReDRAW enables RL agents to be optimized with imagined rollouts under corrected dynamics and then deployed in the real world. In multiple vision-based MuJoCo domains and a physical robot visual lane-following task, ReDRAW effectively models changes to dynamics and avoids overfitting in low data regimes where traditional transfer methods fail.