# Chain of Time: In-Context Physical Simulation with Image Generation Models

arXiv: https://arxiv.org/abs/2511.00110 · HTML: https://mlsanity.com/paper/2511.00110/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 18 of its 43 references that are in the map.
- First posted on arXiv 30 Oct 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 18 of 18)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models](https://mlsanity.com/paper/2406.09403/) (2024)
- [Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters](https://mlsanity.com/paper/2212.10001/) (2022)
- [PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding](https://mlsanity.com/paper/2501.16411/) (2025)
- [Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)](https://mlsanity.com/paper/2402.10376/) (2024)
- [Why think step by step? Reasoning emerges from the locality of experience](https://mlsanity.com/paper/2304.03843/) (2023)
- [Visual Planning: Let's Think Only with Images](https://mlsanity.com/paper/2505.11409/) (2025)
- [Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2412.02104/) (2024)
- [HandRefiner: Refining Malformed Hands in Generated Images by Diffusion-based Conditional Inpainting](https://mlsanity.com/paper/2311.17957/) (2023)
- [PhyBench: A Physical Commonsense Benchmark for Evaluating Text-to-Image Models](https://mlsanity.com/paper/2406.11802/) (2024)
- [An Empirical Study of GPT-4o Image Generation Capabilities](https://mlsanity.com/paper/2504.05979/) (2025)
- [Discovering Failure Modes of Text-guided Diffusion Models via Adversarial Search](https://mlsanity.com/paper/2306.00974/) (2023)
- [Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation](https://mlsanity.com/paper/2506.21876/) (2025)
- [Text-to-Image Diffusion Models Cannot Count, and Prompt Refinement Cannot Help](https://mlsanity.com/paper/2503.06884/) (2025)

## Abstract (arXiv)

We propose a novel cognitively-inspired prompting method to stabilize and interpret the physical simulation produced by autoregressive image generation models, called Chain-of-Time. Chain-of-Time involves generating a series of intermediate images during a simulation, and it is motivated by mental simulation in humans, as well as in-context reasoning in LLMs. Chain-of-Time is used at inference time, and requires no additional fine-tuning. We apply this method to synthetic and real-world domains, including 2-D graphics simulations and natural 3-D videos. These domains cover various physical dynamics, ranging from rigid-body rolling to harmonic oscillation and pulley systems. We found that Chain-of-Time simulation stabilizes the physics simulated by the state-of-the-art image generation models. Beyond examining performance, we also analyzed the specific states of the world simulated by an image model at each time step, which sheds light on the dynamics underlying these simulations. This analysis reveals insights that are hidden from traditional evaluations of physical reasoning, highlighting cases where image generation models can simulate physical properties such as velocity, gravity, oscillations, and pulley systems. Like humans, image generation models are able to accurately predict future world states over short time periods. By providing a mechanism analogous to working memory, Chain-of-Time enables a model to stitch together a series of short-term predictions into an accurate long-term simulation.