# Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics

arXiv: https://arxiv.org/abs/2501.10100 · HTML: https://mlsanity.com/paper/2501.10100/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Placed in Latent dynamics models and model-based RL, one of 11 directions in World models.
- Builds on 38 of its 47 references that are in the map.
- First posted on arXiv 17 Jan 2025.

## Placed in

- World models / [Latent dynamics models and model-based RL](https://mlsanity.com/wm/latent-dynamics-models-and-model-based-rl/)

## Builds on (most cited first, 30 of 38)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [A Generalist Agent](https://mlsanity.com/paper/2205.06175/) (2022)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018)
- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [MOPO: Model-based Offline Policy Optimization](https://mlsanity.com/paper/2005.13239/) (2020)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [TD-MPC2: Scalable, Robust World Models for Continuous Control](https://mlsanity.com/paper/2310.16828/) (2023)
- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [Transformers are Sample-Efficient World Models](https://mlsanity.com/paper/2209.00588/) (2022)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Diffusion for World Modeling: Visual Details Matter in Atari](https://mlsanity.com/paper/2405.12399/) (2024)
- [Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning](https://mlsanity.com/paper/1708.02596/) (2017)
- [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023)
- [Deep Visual Foresight for Planning Robot Motion](https://mlsanity.com/paper/1610.00696/) (2016)
- [Unsupervised Learning for Physical Interaction through Video Prediction](https://mlsanity.com/paper/1605.07157/) (2016)
- [Learning Agile Robotic Locomotion Skills by Imitating Animals](https://mlsanity.com/paper/2004.00784/) (2020)
- [Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control](https://mlsanity.com/paper/1812.00568/) (2018)
- [Accelerated Policy Learning with Parallel Differentiable Simulation](https://mlsanity.com/paper/2204.07137/) (2022)
- [STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learning](https://mlsanity.com/paper/2310.09615/) (2023)
- [Data Efficient Reinforcement Learning for Legged Robots](https://mlsanity.com/paper/1907.03613/) (2019)
- [Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations](https://mlsanity.com/paper/2206.11693/) (2022)

## Built on by (most cited first, 30 of 51)

- [PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation](https://mlsanity.com/paper/2601.03782/) (2026)
- [WorldEval: World Model as Real-World Robot Policies Evaluator](https://mlsanity.com/paper/2505.19017/) (2025)
- [A Comprehensive Survey on World Models for Embodied AI](https://mlsanity.com/paper/2510.16732/) (2025)
- [VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulators](https://mlsanity.com/paper/2510.00406/) (2025)
- [A Survey: Learning Embodied Intelligence from Physical Simulators and World Models](https://mlsanity.com/paper/2507.00917/) (2025)
- [RISE: Self-Improving Robot Policy with Compositional World Model](https://mlsanity.com/paper/2602.11075/) (2026)
- [What about gravity in video generation? Post-Training Newton's Laws with Verifiable Rewards](https://mlsanity.com/paper/2512.00425/) (2025)
- [PlayWorld: Learning Robot World Models from Autonomous Play](https://mlsanity.com/paper/2603.09030/) (2026)
- [Uncertainty-Aware Robotic World Model Makes Offline Model-Based Reinforcement Learning Work on Real Robots](https://mlsanity.com/paper/2504.16680/) (2025)
- [Action Images: End-to-End Policy Learning via Multiview Video Generation](https://mlsanity.com/paper/2604.06168/) (2026)
- [Is the Future Compatible? Diagnosing Dynamic Consistency in World Action Models](https://mlsanity.com/paper/2605.07514/) (2026)
- [Hi-WM: Human-in-the-World-Model for Scalable Robot Post-Training](https://mlsanity.com/paper/2604.21741/) (2026)
- [SkyJEPA: Learning Long-Horizon World Models for Zero-Shot Sim-to-Real Control of Quadrotors](https://mlsanity.com/paper/2606.23444/) (2026)
- [Neural Robot Dynamics](https://mlsanity.com/paper/2508.15755/) (2025)
- [AirScape: An Aerial Generative World Model with Motion Controllability](https://mlsanity.com/paper/2507.08885/) (2025)
- [Ego-Vision World Model for Humanoid Contact Planning](https://mlsanity.com/paper/2510.11682/) (2025)
- [World-Action Models for Robot Learning and Control: A Survey](https://mlsanity.com/paper/2609.16074/) (2026)
- [DreamPolicy: A Unified World-model Policy for Scalable Humanoid Locomotion](https://mlsanity.com/paper/2505.18780/) (2025)
- [Reparameterization Flow Policy Optimization](https://mlsanity.com/paper/2602.03501/) (2026)
- [HAIC: Humanoid Agile Object Interaction Control via Dynamics-Aware World Model](https://mlsanity.com/paper/2602.11758/) (2026)
- [Morphology-Conditioned World Model for Cross-Embodiment Quadrupedal Locomotion](https://mlsanity.com/paper/2604.08780/) (2026)
- [Learning Visual Feature-Based World Models via Residual Latent Action](https://mlsanity.com/paper/2605.07079/) (2026)
- [Safety, Security, and Cognitive Risks in World Models](https://mlsanity.com/paper/2604.01346/) (2026)
- [World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications](https://mlsanity.com/paper/2606.00133/) (2026)
- [SplitAdapter: Load-Aware Humanoid Loco-Manipulation via Factorized Adaptation](https://mlsanity.com/paper/2606.03297/) (2026)
- [MOSAIC: Skill-Centric Manipulation Planning with Physics Simulation](https://mlsanity.com/paper/2504.16738/) (2025)
- [Learning Soft Robotic Dynamics with Active Exploration](https://mlsanity.com/paper/2510.27428/) (2025)
- [Target-Bench: Can Video World Models Achieve Mapless Path Planning with Semantic Targets?](https://mlsanity.com/paper/2511.17792/) (2025)
- [Efficient Model-Based Reinforcement Learning for Robot Control via Online Optimization](https://mlsanity.com/paper/2510.18518/) (2025)
- [Describe-Then-Act: Proactive Agent Steering via Distilled Language-Action World Models](https://mlsanity.com/paper/2603.23149/) (2026)

## Abstract (arXiv)

Learning robust and generalizable world models is crucial for enabling efficient and scalable robotic control in real-world environments. In this work, we introduce a novel framework for learning world models that accurately capture complex, partially observable, and stochastic dynamics. The proposed method employs a dual-autoregressive mechanism and self-supervised training to achieve reliable long-horizon predictions without relying on domain-specific inductive biases, ensuring adaptability across diverse robotic tasks. We further propose a policy optimization framework that leverages world models for efficient training in imagined environments and seamless deployment in real-world systems. This work advances model-based reinforcement learning by addressing the challenges of long-horizon prediction, error accumulation, and sim-to-real transfer. By providing a scalable and robust framework, the introduced methods pave the way for adaptive and efficient robotic systems in real-world applications.