SAMPO:Scale-wise Autoregression with Motion PrOmpt for generative world models
On the map: World models → Video-generation world models and simulators
Abstract (arXiv)
World models allow agents to simulate the consequences of actions in imagined environments for planning, control, and long-horizon decision-making. However, existing autoregressive world models struggle with visually coherent predictions due to disrupted spatial structure, inefficient decoding, and inadequate motion modeling. In response, we propose Scale-wise Autoregression with Motion PrOmpt (SAMPO), a hybrid framework that combines visual autoregressive modeling for intra-frame generation with causal modeling for next-frame generation. Specifically, SAMPO integrates temporal causal decoding with bidirectional spatial attention, which preserves spatial locality and supports parallel decoding within each scale. This design significantly enhances both temporal consistency and rollout efficiency. To further improve dynamic scene understanding, we devise an asymmetric multi-scale tokenizer that preserves spatial details in observed frames and extracts compact dynamic representations for future frames, optimizing both memory usage and model performance. Additionally, we introduce a trajectory-aware motion prompt module that injects spatiotemporal cues about object and robot trajectories, focusing attention on dynamic regions and improving temporal consistency and physical realism. Extensive experiments show that SAMPO achieves competitive performance in action-conditioned video prediction and model-based control, improving generation quality with 4.4× faster inference. We also evaluate SAMPO's zero-shot generalization and scaling behavior, demonstrating its ability to generalize to unseen tasks and benefit from larger model sizes.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 65 of its 80 references that are in the map. First posted on arXiv 19 Sep 2025.
Builds on
65 of 80 references.
Closest ancestors · 6
Same direction, newest first
- 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond6 May
- 2024Genie: Generative Interactive Environments23 Feb
- 2023Learning Interactive Real-World Simulators9 Oct
- 2022Scalable Diffusion Models with Transformers19 Dec
- 2021VideoGPT: Video Generation using VQ-VAE and Transformers20 Apr
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale22 Oct
Other references · 59
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2017Neural Discrete Representation Learningcited by 4,712
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
- 2022Video Diffusion Modelscited by 2,305
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
- 2018World Modelscited by 1,513
- 2018Relational inductive biases, deep learning, and graph networkscited by 1,360
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
- 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
38 more
- 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
- 2024Emu3: Next-Token Prediction is All You Needcited by 781
- 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
- 2023CoTracker: It is Better to Track Togethercited by 522
- 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
- 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
- 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
- 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 474
- 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
- 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 309
- 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
- 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 304
- 2019RoboNet: Large-Scale Multi-Robot Learningcited by 270
- 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 218
- 2024Autoregressive Video Generation without Vector Quantizationcited by 186
- 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 178
- 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
- 2024The pitfalls of next-token predictioncited by 160
- 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 146
- 2017Self-Supervised Visual Planning with Temporal Skip Connectionscited by 139
- 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representationcited by 123
- 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 114
- 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 108
- 2024MotionBooth: Motion-Aware Customized Text-to-Video Generationcited by 93
- 2019Unsupervised Learning of Object Structure and Dynamics from Videoscited by 92
- 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitationcited by 90
- 2019High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networkscited by 88
- 2021FitVid: Overfitting in Pixel-Level Video Predictioncited by 87
- 2023Fine-Grained Visual Promptingcited by 82
- 2021Greedy Hierarchical Variational Autoencoders for Large-Scale Video Predictioncited by 75
- 2023Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learningcited by 45
- 2025VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Modelcited by 44
- 2025Pre-Trained Video Generative Models as World Simulatorscited by 44
- 2025ReconDreamer++: Harmonizing Generative and Reconstructive Models for Driving Scene Representationcited by 36
- 2025FastVAR: Linear Visual Autoregressive Modeling via Cached Token Pruningcited by 34
- 2024ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Modelscited by 33
- 2025Taming Teacher Forcing for Masked Autoregressive Video Generationcited by 30
50 most cited of 59 All 59 (JSON)
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026GigaWorld-Policy: An Efficient Action-Centered World--Action Modelcited by 139
- 2026VAG: Dual-Stream Video-Action Generation for Embodied Data Synthesiscited by 6
- 2026Learning Transferable Dynamics Priors from Action to World Modelingcited by 6
- 2026MCP-Cosmos: World Model-Augmented Agents for Complex Task Execution in MCP Environmentscited by 1
- 2026Learning Transferable Dynamics Priors from Action to World Modeling28 Jun 2026
- 2026MCP-Cosmos: World Model-Augmented Agents for Complex Task Execution in MCP Environments9 May 2026
- 2026VAG: Dual-Stream Video-Action Generation for Embodied Data Synthesis10 Apr 2026
- 2026GigaWorld-Policy: An Efficient Action-Centered World--Action Model18 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2025WorldVLA: Towards Autoregressive Action World Model
- 2026RAYNOVA: Scale-Temporal Autoregressive World Modeling in Ray Space
- 2026WALL-SS: Scaling Long-horizon World Models via Next-Scale Autoregression
- 2026CSWAM: Better Causal Semantic Representations for Out-of-Distribution Generalization in World Action Models
- 2026World-T3: Test-Time Training as Persistent Memory for Real-Time Video World Models
- 2026The Past Frames the Future: Memory for Autoregressive Video Generation
- 2026DriveVA: Video Action Models are Zero-Shot Drivers
- 2025Visual Self-Refinement for Autoregressive Models