# ChronoDreamer: Action-Conditioned World Model as an Online Simulator for Robotic Planning

arXiv: https://arxiv.org/abs/2512.18619 · HTML: https://mlsanity.com/paper/2512.18619/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 21 Dec 2025.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)

## Builds on (most cited first, 13 of 13)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [RLBench: The Robot Learning Benchmark & Learning Environment](https://mlsanity.com/paper/1909.12271/) (2019)
- [Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Prediction](https://mlsanity.com/paper/2003.01460/) (2020)
- [PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generation](https://mlsanity.com/paper/2409.18964/) (2024)
- [Action Noise in Off-Policy Deep Reinforcement Learning: Impact on Exploration and Performance](https://mlsanity.com/paper/2206.03787/) (2022)
- [FreeTacMan: Robot-free Visuo-Tactile Data Collection System for Contact-rich Manipulation](https://mlsanity.com/paper/2506.01941/) (2025)
- [WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planning](https://mlsanity.com/paper/2506.04363/) (2025)
- [Neural Contact Fields: Tracking Extrinsic Contact with Tactile Sensing](https://mlsanity.com/paper/2210.09297/) (2022)
- [Enhancing Physical Plausibility in Video Generation by Reasoning the Implausibility](https://mlsanity.com/paper/2509.24702/) (2025)
- [DyTact: Capturing Dynamic Contacts in Hand-Object Manipulation](https://mlsanity.com/paper/2506.03103/) (2025)

## Built on by (most cited first, 4 of 4)

- [MVISTA-4D: View-Consistent 4D World Model with Test-Time Action Inference for Robotic Manipulation](https://mlsanity.com/paper/2602.09878/) (2026)
- [Physically Viable World Models: A Case for Query-Conditioned Embodied AI](https://mlsanity.com/paper/2605.30542/) (2026)
- [From Generation to Simulation: How Far Are World Models from Being True Simulators?](https://mlsanity.com/paper/2608.23070/) (2026)
- [CAER: Causal Action Effect Reweighting for World Model Training](https://mlsanity.com/paper/2608.30897/) (2026)

## Abstract (arXiv)

We present ChronoDreamer, an action-conditioned world model for contact-rich robotic manipulation. Given a history of egocentric RGB frames, contact maps, actions, and joint states, ChronoDreamer predicts future video frames, contact distributions, and joint angles via a spatial-temporal transformer trained with MaskGIT-style masked prediction. Contact is encoded as depth-weighted Gaussian splat images that render 3D forces into a camera-aligned format suitable for vision backbones. At inference, predicted rollouts are evaluated by a vision-language model that reasons about collision likelihood, enabling rejection sampling of unsafe actions before execution. We train and evaluate on DreamerBench, a simulation dataset generated with Project Chrono that provides synchronized RGB, contact splat, proprioception, and physics annotations across rigid and deformable object scenarios. Qualitative results demonstrate that the model preserves spatial coherence during non-contact motion and generates plausible contact predictions, while the LLM-based judge distinguishes collision from non-collision trajectories.