# Sekai: A Video Dataset towards World Exploration

arXiv: https://arxiv.org/abs/2506.15675 · HTML: https://mlsanity.com/paper/2506.15675/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2025.

## Built on by (most cited first, 30 of 67)

- [Matrix-game 2.0: An open-source real-time and streaming interactive world model](https://mlsanity.com/paper/2508.13009/) (2025)
- [WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling](https://mlsanity.com/paper/2512.14614/) (2025)
- [Emu3.5: Native Multimodal Models are World Learners](https://mlsanity.com/paper/2510.26583/) (2025)
- [RELIC: Interactive Video World Model with Long-Horizon Memory](https://mlsanity.com/paper/2512.04040/) (2025)
- [Yume: An Interactive World Generation Model](https://mlsanity.com/paper/2507.17744/) (2025)
- [SpatialVID: A Large-Scale Video Dataset with Spatial Annotations](https://mlsanity.com/paper/2509.09676/) (2025)
- [OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling](https://mlsanity.com/paper/2509.12201/) (2025)
- [Context Forcing: Consistent Autoregressive Video Generation with Long Context](https://mlsanity.com/paper/2602.06028/) (2026)
- [SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer](https://mlsanity.com/paper/2605.15178/) (2026)
- [Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memory](https://mlsanity.com/paper/2602.02393/) (2026)
- [DreamX-World 1.0: A General-Purpose Interactive World Model](https://mlsanity.com/paper/2606.16993/) (2026)
- [Astra: General Interactive World Model with Autoregressive Denoising](https://mlsanity.com/paper/2512.08931/) (2025)
- [Pack and Force Your Memory: Long-form and Consistent Video Generation](https://mlsanity.com/paper/2510.01784/) (2025)
- [VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Control](https://mlsanity.com/paper/2601.05138/) (2026)
- [MosaicMem: Hybrid Spatial Memory for Controllable Video World Models](https://mlsanity.com/paper/2603.17117/) (2026)
- [Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models](https://mlsanity.com/paper/2603.25716/) (2026)
- [VGGT-Omega](https://mlsanity.com/paper/2605.15195/) (2026)
- [Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Models](https://mlsanity.com/paper/2603.22212/) (2026)
- [WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPG](https://mlsanity.com/paper/2603.23497/) (2026)
- [Mode Seeking meets Mean Seeking for Fast Long Video Generation](https://mlsanity.com/paper/2602.24289/) (2026)
- [WorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representation](https://mlsanity.com/paper/2603.16871/) (2026)
- [SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models](https://mlsanity.com/paper/2609.02886/) (2026)
- [AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report](https://mlsanity.com/paper/2607.18367/) (2026)
- [ReWorld: An Interactive World Model with Long-Horizon Memory](https://mlsanity.com/paper/2608.23565/) (2026)
- [MemLearner: Learning to Query Context memory for Video World Models](https://mlsanity.com/paper/2606.31734/) (2026)
- [MVInverse: Feed-forward Multi-view Inverse Rendering in Seconds](https://mlsanity.com/paper/2512.21003/) (2025)
- [BiWM: Advancing Open-Source Interactive Video World Models with Bidirectional Autoregression](https://mlsanity.com/paper/2606.10135/) (2026)
- [OmniShotCut: Holistic Relational Shot Boundary Detection with Shot-Query Transformer](https://mlsanity.com/paper/2604.24762/) (2026)
- [Worldscape-MoE: A Unified Mixture-of-Experts World Model for Scalable Heterogeneous Action Control](https://mlsanity.com/paper/2607.03964/) (2026)
- [TrajVG: 3D Trajectory-Coupled Visual Geometry Learning](https://mlsanity.com/paper/2602.04439/) (2026)

## Abstract (arXiv)

Video generation techniques have made remarkable progress, promising to be the foundation of interactive world exploration. However, existing video generation datasets are not well-suited for world exploration training as they suffer from some limitations: limited locations, short duration, static scenes, and a lack of annotations about exploration and the world. In this paper, we introduce Sekai (meaning "world" in Japanese), a high-quality first-person view worldwide video dataset with rich annotations for world exploration. It consists of over 5,000 hours of walking or drone view (FPV and UVA) videos from over 100 countries and regions across 750 cities. We develop an efficient and effective toolbox to collect, pre-process and annotate videos with location, scene, weather, crowd density, captions, and camera trajectories. Comprehensive analyses and experiments demonstrate the dataset's scale, diversity, annotation quality, and effectiveness for training video generation models. We believe Sekai will benefit the area of video generation and world exploration, and motivate valuable applications. The project page is https://lixsp11.github.io/sekai-project/.