# AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video

arXiv: https://arxiv.org/abs/2609.14462 · HTML: https://mlsanity.com/paper/2609.14462/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 40 of its 69 references that are in the map.
- First posted on arXiv 13 Sep 2026.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)

## Builds on (most cited first, 30 of 40)

- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [CameraCtrl: Enabling Camera Control for Text-to-Video Generation](https://mlsanity.com/paper/2404.02101/) (2024)
- [Self-Forcing++: Towards Minute-Scale High-Quality Video Generation](https://mlsanity.com/paper/2510.02283/) (2025)
- [TransNet V2: An effective deep network architecture for fast shot transition detection](https://mlsanity.com/paper/2008.04838/) (2020)
- [ViPE: Video Pose Engine for 3D Geometric Perception](https://mlsanity.com/paper/2508.10934/) (2025)
- [CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generation](https://mlsanity.com/paper/2406.02509/) (2024)
- [Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory](https://mlsanity.com/paper/2604.08995/) (2026)
- [RELIC: Interactive Video World Model with Long-Horizon Memory](https://mlsanity.com/paper/2512.04040/) (2025)
- [HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds](https://mlsanity.com/paper/2604.14268/) (2026)
- [Imagine360: Immersive 360 Video Generation from Perspective Anchor](https://mlsanity.com/paper/2412.03552/) (2024)
- [PanoWan: Lifting Diffusion Video Generation Models to 360deg with Latitude/Longitude-aware Mechanisms](https://mlsanity.com/paper/2505.22016/) (2025)
- [Latent Spatial Memory for Video World Models](https://mlsanity.com/paper/2606.09828/) (2026)
- [OmniRoam: World Wandering via Long-Horizon Panoramic Video Generation](https://mlsanity.com/paper/2603.30045/) (2026)
- [Wonder: Video World Model Done Better](https://mlsanity.com/paper/2607.26037/) (2026)
- [VideoPanda: Video Panoramic Diffusion with Multi-view Attention](https://mlsanity.com/paper/2504.11389/) (2025)
- [Alaya-EVOKE: From Linear-Scaling Supervision to Endless World](https://mlsanity.com/paper/2608.13546/) (2026)
- [Generative World Renderer](https://mlsanity.com/paper/2604.02329/) (2026)
- [AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report](https://mlsanity.com/paper/2607.18367/) (2026)
- [CubeComposer: Spatio-Temporal Autoregressive 4K 360deg Video Generation from Perspective Video](https://mlsanity.com/paper/2603.04291/) (2026)
- [ReWorld: An Interactive World Model with Long-Horizon Memory](https://mlsanity.com/paper/2608.23565/) (2026)
- [Leader360V: The Large-scale, Real-world 360 Video Dataset for Multi-task Learning in Diverse Environment](https://mlsanity.com/paper/2506.14271/) (2025)
- [CamPVG: Camera-Controlled Panoramic Video Generation with Epipolar-Aware Diffusion](https://mlsanity.com/paper/2509.19979/) (2025)
- [Holo360D: A Large-Scale Real-World Dataset with Continuous Trajectories for Advancing Panoramic 3D Reconstruction and Beyond](https://mlsanity.com/paper/2604.22482/) (2026)
- [PanoWorld-X: Generating Explorable Panoramic Worlds via Sphere-Aware Video Diffusion](https://mlsanity.com/paper/2509.24997/) (2025)
- [AlayaWorld: Interactive Long-Horizon World Modeling - Full Technical Report (v1.1)](https://mlsanity.com/paper/2608.13492/) (2026)
- [Generating 360deg Video is What You Need For a 3D Scene](https://mlsanity.com/paper/2504.02045/) (2025)
- [PanoWorld: Real-World Panoramic Generation](https://mlsanity.com/paper/2607.09661/) (2026)
- [Generative World Renderer at the Speed of Play](https://mlsanity.com/paper/2607.18703/) (2026)
- [SphereDiff: Tuning-free 360deg Static and Dynamic Panorama Generation via Spherical Latent Representation](https://mlsanity.com/paper/2504.14396/) (2025)

## Built on by (most cited first, 3 of 3)

- [MUGEN: Interactive Panoramic World Exploration via Camera Control](https://mlsanity.com/paper/2609.38077/) (2026)
- [SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation](https://mlsanity.com/paper/2610.08941/) (2026)
- [PAWA: Panoramic Action-conditioned World Modeling for Multiple Agents](https://mlsanity.com/paper/2610.20240/) (2026)

## Abstract (arXiv)

Interactive video world models must maintain broad scene context under camera motion while producing high-fidelity observations with low latency. Existing approaches face a representation trade-off: perspective models operate on local views and must preserve off-screen content over long rollouts, whereas broader spatial coverage is typically obtained by synthesizing full-sphere videos or constructing explicit 3D representations. Motivated by the complementary roles of global context and selective local acuity in visual perception, we present AlayaVista, a camera-controllable streaming video world model that decouples panoramic world evolution from perspective observation synthesis. Given a single perspective image, AlayaVista constructs a 360-degree scene prior using a pretrained panorama expansion model and then evolves the scene as a camera-conditioned panoramic latent state. A latent viewport renderer maps this state to the requested perspective video latents, while a perspective refiner restores details, suppresses artifacts, and performs super-resolution. To support efficient streaming, we adapt the panoramic generator to chunk-autoregressive generation and distill both panoramic generation and perspective refinement into few-step processes. To provide the supervision required by this design, we construct MUGEN, a large-scale real-world panoramic video dataset containing 1,318 hours of videos at resolutions of at least 4K, together with rich semantic and geometric annotations.