# Directing the World: Fast Autoregressive Video Generation with Compositional Human-Camera Control arXiv: https://arxiv.org/abs/2606.27964 ยท HTML: https://mlsanity.com/paper/2606.27964/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Placed in Video-generation world models and simulators, one of 11 directions in World models. - Builds on 34 of its 51 references that are in the map. - First posted on arXiv 26 Jun 2026. ## Placed in - World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/) ## Builds on (most cited first, 30 of 34) - [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025) - [Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion](https://mlsanity.com/paper/2506.08009/) (2025) - [VideoPoet: A Large Language Model for Zero-Shot Video Generation](https://mlsanity.com/paper/2312.14125/) (2023) - [Phenaki: Variable Length Video Generation From Open Domain Textual Description](https://mlsanity.com/paper/2210.02399/) (2022) - [CameraCtrl: Enabling Camera Control for Text-to-Video Generation](https://mlsanity.com/paper/2404.02101/) (2024) - [From Slow Bidirectional to Fast Autoregressive Video Diffusion Models](https://mlsanity.com/paper/2412.07772/) (2024) - [Causal World Modeling for Robot Control](https://mlsanity.com/paper/2601.21998/) (2026) - [Effective Whole-body Pose Estimation with Two-stages Distillation](https://mlsanity.com/paper/2307.15880/) (2023) - [MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance](https://mlsanity.com/paper/2406.19680/) (2024) - [Autoregressive Video Generation without Vector Quantization](https://mlsanity.com/paper/2412.14169/) (2024) - [Advancing Open-source World Models](https://mlsanity.com/paper/2601.20540/) (2026) - [Long-Context Autoregressive Video Modeling with Next-Frame Prediction](https://mlsanity.com/paper/2503.19325/) (2025) - [CamI2V: Camera-Controlled Image-to-Video Diffusion Model](https://mlsanity.com/paper/2410.15957/) (2024) - [CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models](https://mlsanity.com/paper/2503.10592/) (2025) - [RELIC: Interactive Video World Model with Long-Horizon Memory](https://mlsanity.com/paper/2512.04040/) (2025) - [Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generation](https://mlsanity.com/paper/2504.14899/) (2025) - [Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharing](https://mlsanity.com/paper/2411.16375/) (2024) - [Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising](https://mlsanity.com/paper/2604.26694/) (2026) - [UltraVideo: High-Quality UHD Video Dataset with Comprehensive Captions](https://mlsanity.com/paper/2506.13691/) (2025) - [Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance](https://mlsanity.com/paper/2512.08765/) (2025) - [VRAG: Learning World Models for Interactive Video Generation](https://mlsanity.com/paper/2505.21996/) (2025) - [Cascade-Zero123: One Image to Highly Consistent 3D with Self-Prompted Nearby Views](https://mlsanity.com/paper/2312.04424/) (2023) - [Astra: General Interactive World Model with Autoregressive Denoising](https://mlsanity.com/paper/2512.08931/) (2025) - [RealisDance-DiT: Simple yet Strong Baseline towards Controllable Character Animation in the Wild](https://mlsanity.com/paper/2504.14977/) (2025) - [Macro-from-Micro Planning for High-Quality and Parallelized Autoregressive Long Video Generation](https://mlsanity.com/paper/2508.03334/) (2025) - [DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving](https://mlsanity.com/paper/2601.01528/) (2026) - [VideoMAR: Autoregressive Video Generatio with Continuous Tokens](https://mlsanity.com/paper/2506.14168/) (2025) - [TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model](https://mlsanity.com/paper/2601.00051/) (2025) - [LIVE: Long-horizon Interactive Video World Modeling](https://mlsanity.com/paper/2602.03747/) (2026) - [LiftImage3D: Lifting Any Single Image to 3D Gaussians with Video Generation Priors](https://mlsanity.com/paper/2412.09597/) (2024) ## Built on by (most cited first, 6 of 6) - [CineWeaver: Training-Free Reference-Controllable Multi-Shot Long Video Generation for Cinematic Storytelling](https://mlsanity.com/paper/2607.26529/) (2026) - [TourPhysics: Bringing Physics to World Models for Exploration and Manipulation from a Single Image](https://mlsanity.com/paper/2609.04911/) (2026) - [Search-to-World: Evaluation of 3D World Delivery from User Request through Web Search](https://mlsanity.com/paper/2609.07605/) (2026) - [Uncertainty DMD: Restoring Diversity in Few-Step Autoregressive Video Distillation](https://mlsanity.com/paper/2609.11265/) (2026) - [World in World: Explore the World with World Models](https://mlsanity.com/paper/2609.11548/) (2026) - [DiVid: Diagnosing Dimension-Specific Diversity Collapse in Video Generation Models](https://mlsanity.com/paper/2610.01661/) (2026) ## Abstract (arXiv) Building interactive world models requires generating realistic videos while maintaining controllable dynamics over long horizons. Autoregressive video generation offers a scalable foundation, but suffers from error accumulation and temporal degradation during extended rollouts. This issue is further amplified under heterogeneous controls such as human motion and camera trajectories, which may interfere and destabilize a pretrained video prior, while existing methods often trade off controllability and visual quality. We propose "Directing the World", a fast autoregressive framework for controllable world-model video generation with compositional human-motion and camera-trajectory control. Our key idea is to decouple control learning while preserving a unified autoregressive video prior. We introduce a Fast-Slow Memory training strategy to stabilize long-horizon rollout learning and improve convergence. For human motion control, we design a t-guided Dynamic Projection mechanism and a refined Motion-CFG strategy, enabling temporally smooth and accurate motion alignment without degrading visual fidelity, and supporting multi-person control.After learning a robust motion prior, we introduce a second-stage camera-trajectory control module to compose human dynamics with viewpoint changes for coherent world exploration. We further construct a large-scale dataset with synchronized video, text, human-motion, and camera-trajectory annotations, organized into motion-centric and camera-centric subsets for decoupled training. Extensive experiments show stable long-horizon generation with precise controllability and high visual quality. See more at https://whydahuzi.github.io/Directing-the-World.github.io/.