mlsanityData to 7 Oct 2026

RAYNOVA: Scale-Temporal Autoregressive World Modeling in Ray Space

arXiv 2602.20685 ↗First posted 24 Feb 2026cs.CVcited by 4

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

World foundation models aim to simulate the evolution of the real world with physically plausible behavior. Unlike prior methods that handle spatial and temporal correlations separately, we propose RAYNOVA, a geometry-agonistic multiview world model for driving scenarios that employs a dual-causal autoregressive framework. It follows both scale-wise and temporal topological orders in the autoregressive process, and leverages global attention for unified 4D spatio-temporal reasoning. Different from existing works that impose strong 3D geometric priors, RAYNOVA constructs an isotropic spatio-temporal representation across views, frames, and scales based on relative Pl"ucker-ray positional encoding, enabling robust generalization to diverse camera setups and ego motions. We further introduce a recurrent training paradigm to alleviate distribution drift in long-horizon video generation. RAYNOVA achieves state-of-the-art multi-view video generation results on nuScenes, while offering higher throughput and strong controllability under diverse input conditions, generalizing to novel views and camera configurations without explicit 3D scene representation. Our code will be released at https://raynova-ai.github.io/.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 66 of its 78 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

66 of 78 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024WorldSimBench: Towards Video Generation Models as World Simulators23 Oct
  2. 2024Genie: Generative Interactive Environments23 Feb
  3. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation6 Feb
  4. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  5. 2022MAGVIT: Masked Generative Video Transformer10 Dec
  6. 2021VideoGPT: Video Generation using VQ-VAE and Transformers20 Apr

Other references · 60

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  8. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  9. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  10. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  11. 2018World Modelscited by 1,444
  12. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
38 more
  1. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  2. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  3. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  4. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  5. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  6. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  7. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  8. 2024Movie Gen: A Cast of Media Foundation Modelscited by 614
  9. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  10. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  11. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  12. 2023DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenescited by 359
  13. 2022Transformers are Sample-Efficient World Modelscited by 317
  14. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
  15. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  16. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  17. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 284
  18. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 258
  19. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 250
  20. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  21. 2024Rotary Position Embedding for Vision Transformercited by 226
  22. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
  23. 2024Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splattingcited by 218
  24. 2024LVSM: A Large View Synthesis Model with Minimal 3D Inductive Biascited by 178
  25. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 178
  26. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 165
  27. 2024OmniRe: Omni Urban Scene Reconstructioncited by 159
  28. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoningcited by 142
  29. 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animationcited by 101
  30. 2023WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generationcited by 93
  31. 2022Object Scene Representation Transformercited by 90
  32. 2023SparseFusion: Fusing Multi-Modal Sparse Representations for Multi-Sensor 3D Object Detectioncited by 83
  33. 2025RayZer: A Self-supervised Large View Synthesis Modelcited by 74
  34. 2024MagicDrive3D: Controllable 3D Generation for Any-View Rendering in Street Scenescited by 70
  35. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 66
  36. 2023ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modelingcited by 60
  37. 2025HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generationcited by 56
  38. 2024FreeVS: Generative View Synthesis on Free Driving Trajectorycited by 52

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Video World Models with Long-term Spatial Memory
  2. 2025SAMPO:Scale-wise Autoregression with Motion PrOmpt for generative world models
  3. 2026AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video
  4. 2025DeepVerse: 4D Autoregressive Video Generation as a World Model
  5. 2026VGGT-World: Transforming VGGT into an Autoregressive Geometry World Model
  6. 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modeling
  7. 2025LongVie 2: Multimodal Controllable Ultra-Long Video World Model
  8. 2025WorldPack: Dynamic Frame Compression for Long-context Video World Modeling