Self-Supervised Multi-Modal World Model with 4D Space-Time Embedding
On the map: World models → Video-generation world models and simulators
Abstract (arXiv)
We present DeepEarth, a self-supervised multi-modal world model with Earth4D, a novel planetary-scale 4D space-time positional encoder. Earth4D extends 3D multi-resolution hash encoding to include time, efficiently scaling across the planet over centuries with sub-meter, sub-second precision. Multi-modal encoders (e.g. vision-language models) are fused with Earth4D embeddings and trained via masked reconstruction. We demonstrate Earth4D's expressive power by achieving state-of-the-art performance on an ecological forecasting benchmark. Earth4D with learnable hash probing surpasses a multi-modal foundation model pre-trained on substantially more data. Access open source code and download models at: https://github.com/legel/deepearth
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 8 of its 13 references that are in the map. First posted on arXiv 7 Mar 2026.
Builds on
8 of 13 references.
Closest ancestors · 1
Same direction, newest first
Other references · 7
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
- 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
- 2025AlphaEarth Foundations: An embedding field model for accurate and efficient global mapping from sparse label datacited by 122
- 2024Grid4D: 4D Decomposed Hash Encoding for High-Fidelity Dynamic Gaussian Splattingcited by 49
- 2025Galileo: Learning Global & Local Features of Many Remote Sensing Modalitiescited by 41
- 2023Compact Neural Graphics Primitives with Learned Hash Probingcited by 19
- 2025High-Resolution Live Fuel Moisture Content (LFMC) Maps for Wildfire Risk from Multimodal Earth Observation Datacited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling
- 2025Terra: Explorable Native 3D World Model with Point Latents
- 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion
- 2026Emergent Semantic Representations in World Models through Physical Interaction without Linguistic Supervision
- 2026HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds
- 2026LongEarth-R1: Benchmarking and Aligning Vision-Language Models for Long-Horizon Earth Observation Reasoning
- 2024D2-World: An Efficient World Model through Decoupled Dynamic Flow
- 2025Simple, Good, Fast: Self-Supervised World Models Free of Baggage