mlsanityData to 14 Oct 2026

SAMPO:Scale-wise Autoregression with Motion PrOmpt for generative world models

arXiv 2509.15536 ↗First posted 19 Sep 2025cs.CV cs.ROcited by 4

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

World models allow agents to simulate the consequences of actions in imagined environments for planning, control, and long-horizon decision-making. However, existing autoregressive world models struggle with visually coherent predictions due to disrupted spatial structure, inefficient decoding, and inadequate motion modeling. In response, we propose Scale-wise Autoregression with Motion PrOmpt (SAMPO), a hybrid framework that combines visual autoregressive modeling for intra-frame generation with causal modeling for next-frame generation. Specifically, SAMPO integrates temporal causal decoding with bidirectional spatial attention, which preserves spatial locality and supports parallel decoding within each scale. This design significantly enhances both temporal consistency and rollout efficiency. To further improve dynamic scene understanding, we devise an asymmetric multi-scale tokenizer that preserves spatial details in observed frames and extracts compact dynamic representations for future frames, optimizing both memory usage and model performance. Additionally, we introduce a trajectory-aware motion prompt module that injects spatiotemporal cues about object and robot trajectories, focusing attention on dynamic regions and improving temporal consistency and physical realism. Extensive experiments show that SAMPO achieves competitive performance in action-conditioned video prediction and model-based control, improving generation quality with 4.4× faster inference. We also evaluate SAMPO's zero-shot generalization and scaling behavior, demonstrating its ability to generalize to unseen tasks and benefit from larger model sizes.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 65 of its 80 references that are in the map. First posted on arXiv 19 Sep 2025.

Builds on

65 of 80 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond6 May
  2. 2024Genie: Generative Interactive Environments23 Feb
  3. 2023Learning Interactive Real-World Simulators9 Oct
  4. 2022Scalable Diffusion Models with Transformers19 Dec
  5. 2021VideoGPT: Video Generation using VQ-VAE and Transformers20 Apr
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale22 Oct

Other references · 59

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2017Neural Discrete Representation Learningcited by 4,712
  4. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  5. 2022Video Diffusion Modelscited by 2,305
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  7. 2018World Modelscited by 1,513
  8. 2018Relational inductive biases, deep learning, and graph networkscited by 1,360
  9. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  10. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  11. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  12. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
38 more
  1. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  2. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  3. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  4. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  5. 2023CoTracker: It is Better to Track Togethercited by 522
  6. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  7. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
  8. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  9. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 474
  10. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  11. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 309
  12. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
  13. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 304
  14. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 270
  15. 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 218
  16. 2024Autoregressive Video Generation without Vector Quantizationcited by 186
  17. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 178
  18. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  19. 2024The pitfalls of next-token predictioncited by 160
  20. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 146
  21. 2017Self-Supervised Visual Planning with Temporal Skip Connectionscited by 139
  22. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representationcited by 123
  23. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 114
  24. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 108
  25. 2024MotionBooth: Motion-Aware Customized Text-to-Video Generationcited by 93
  26. 2019Unsupervised Learning of Object Structure and Dynamics from Videoscited by 92
  27. 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitationcited by 90
  28. 2019High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networkscited by 88
  29. 2021FitVid: Overfitting in Pixel-Level Video Predictioncited by 87
  30. 2023Fine-Grained Visual Promptingcited by 82
  31. 2021Greedy Hierarchical Variational Autoencoders for Large-Scale Video Predictioncited by 75
  32. 2023Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learningcited by 45
  33. 2025VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Modelcited by 44
  34. 2025Pre-Trained Video Generative Models as World Simulatorscited by 44
  35. 2025ReconDreamer++: Harmonizing Generative and Reconstructive Models for Driving Scene Representationcited by 36
  36. 2025FastVAR: Linear Visual Autoregressive Modeling via Cached Token Pruningcited by 34
  37. 2024ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Modelscited by 33
  38. 2025Taming Teacher Forcing for Masked Autoregressive Video Generationcited by 30

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025WorldVLA: Towards Autoregressive Action World Model
  2. 2026RAYNOVA: Scale-Temporal Autoregressive World Modeling in Ray Space
  3. 2026WALL-SS: Scaling Long-horizon World Models via Next-Scale Autoregression
  4. 2026CSWAM: Better Causal Semantic Representations for Out-of-Distribution Generalization in World Action Models
  5. 2026World-T3: Test-Time Training as Persistent Memory for Real-Time Video World Models
  6. 2026The Past Frames the Future: Memory for Autoregressive Video Generation
  7. 2026DriveVA: Video Action Models are Zero-Shot Drivers
  8. 2025Visual Self-Refinement for Autoregressive Models