mlsanityData to 14 Oct 2026

Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model

arXiv 2502.10248 ↗First posted 14 Feb 2025cs.CV cs.CLcited by 168

Not on the map. Why?

Cited by 64 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present Step-Video-T2V, a state-of-the-art text-to-video pre-trained model with 30B parameters and the ability to generate videos up to 204 frames in length. A deep compression Variational Autoencoder, Video-VAE, is designed for video generation tasks, achieving 16x16 spatial and 8x temporal compression ratios, while maintaining exceptional video reconstruction quality. User prompts are encoded using two bilingual text encoders to handle both English and Chinese. A DiT with 3D full attention is trained using Flow Matching and is employed to denoise input noise into latent frames. A video-based DPO approach, Video-DPO, is applied to reduce artifacts and improve the visual quality of the generated videos. We also detail our training strategies and share key observations and insights. Step-Video-T2V's performance is evaluated on a novel video generation benchmark, Step-Video-T2V-Eval, demonstrating its state-of-the-art text-to-video quality when compared with both open-source and commercial engines. Additionally, we discuss the limitations of current diffusion-based model paradigm and outline future directions for video foundation models. We make both Step-Video-T2V and Step-Video-T2V-Eval available at https://github.com/stepfun-ai/Step-Video-T2V. The online version can be accessed from https://yuewen.cn/videos as well. Our goal is to accelerate the innovation of video foundation models and empower video content creators.

Key facts. Cited by 168 papers in the map (data through 14 Oct 2026). Builds on 37 of its 47 references that are in the map. First posted on arXiv 14 Feb 2025.

Builds on

37 of 47 references.

References · 37

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  7. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  9. 2017Deep reinforcement learning from human preferencescited by 3,912
  10. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  11. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  12. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
25 more
  1. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  2. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  3. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  4. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
  5. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
  6. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 652
  7. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
  8. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 632
  9. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
  10. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 457
  11. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 393
  12. 2022Reducing Activation Recomputation in Large Transformer Modelscited by 302
  13. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 298
  14. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
  15. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 279
  16. 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 252
  17. 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 245
  18. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 217
  19. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 196
  20. 2022Chinese CLIP: Contrastive Vision-Language Pretraining in Chinesecited by 109
  21. 2024Improving the Training of Rectified Flowscited by 108
  22. 2024WF-VAE: Enhancing Video VAE by Wavelet-Driven Energy Flow for Latent Video Diffusion Modelcited by 57
  23. 2025Taming Teacher Forcing for Masked Autoregressive Video Generationcited by 30
  24. 2024DistTrain: Addressing Model and Data Heterogeneity with Disaggregated Training for Multimodal Large Language Modelscited by 16
  25. 2025DSV: Exploiting Dynamic Sparsity to Accelerate Large-Scale Video DiT Trainingcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 100; 2026: 68.

168 papers.

Sort:

By area:

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  2. 2025MAGI-1: Autoregressive Video Generation at Scalecited by 276
  3. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
  4. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 184
  5. 2025Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200kcited by 163
  6. 2025HunyuanVideo 1.5 Technical Reportcited by 160
  7. 2025VSA: Faster Video Diffusion with Trainable Sparse Attentioncited by 104
  8. 2025CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Modelscited by 98
  9. 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Modelscited by 98
  10. 2025Controllable Video Generation: A Surveycited by 98
  11. 2025Lumina-Image 2.0: A Unified and Efficient Image Generative Frameworkcited by 96
  12. 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformercited by 94
38 more
  1. 2025WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generationcited by 90
  2. 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assetscited by 87
  3. 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 87
  4. 2025LongCat-Video Technical Reportcited by 85
  5. 2025Seaweed-7B: Cost-Effective Training of Video Generation Foundation Modelcited by 82
  6. 2025Scaling Instruction-Based Video Editing with a High-Quality Synthetic Datasetcited by 79
  7. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  8. 2025RewardDance: Reward Scaling in Visual Generationcited by 68
  9. 2026Helios: Real Real-Time Long Video Generation Modelcited by 65
  10. 2025Long-Context State-Space Video World Modelscited by 64
  11. 2025UniVerse-1: Unified Audio-Video Generation via Stitching of Expertscited by 61
  12. 2025UltraVideo: High-Quality UHD Video Dataset with Comprehensive Captionscited by 57
  13. 2025NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scalecited by 57
  14. 2025Waver: Wave Your Way to Lifelike Video Generationcited by 52
  15. 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generationcited by 49
  16. 2025MiniMax-Remover: Taming Bad Noise Helps Video Object Removalcited by 45
  17. 2025Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidancecited by 45
  18. 2025ATI: Any Trajectory Instruction for Controllable Video Generationcited by 43
  19. 2025Video-T1: Test-Time Scaling for Video Generationcited by 40
  20. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 40
  21. 2026Qwen-Image-2.0 Technical Reportcited by 36
  22. 2025Training-Free Efficient Video Generation via Dynamic Token Carvingcited by 34
  23. 2025DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Modelscited by 34
  24. 2025EX-4D: EXtreme Viewpoint 4D Video Synthesis via Depth Watertight Meshcited by 32
  25. 2025OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive Simulationcited by 32
  26. 2025EchoShot: Multi-Shot Portrait Video Generationcited by 29
  27. 2026OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoningcited by 27
  28. 2026VINO: A Unified Visual Generator with Interleaved OmniModal Contextcited by 26
  29. 2025LoViC: Efficient Long Video Generation with Context Compressioncited by 19
  30. 2025Bidirectional Sparse Attention for Faster Video Diffusion Trainingcited by 19
  31. 2025FreeLong++: Training-Free Long Video Generation via Multi-band SpectralFusioncited by 17
  32. 2026PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Modelscited by 17
  33. 2025FiVE: A Fine-grained Video Editing Benchmark for Evaluating Emerging Diffusion and Rectified Flow Modelscited by 16
  34. 2025Generative Pre-trained Autoregressive Diffusion Transformercited by 16
  35. 2025DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencodercited by 16
  36. 2026ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPUcited by 16
  37. 2025Audio-Sync Video Generation with Multi-Stream Temporal Controlcited by 15
  38. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15

50 most cited of 168 All 168 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024xGen-VideoSyn-1: High-fidelity Text-to-Video Synthesis with Compressed Representations
  2. 2024CV-VAE: A Compatible Video VAE for Latent Generative Video Models
  3. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generation
  4. 2023A Recipe for Scaling up Text-to-Video Generation with Text-free Videos
  5. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation
  6. 2024Searching Priors Makes Text-to-Video Synthesis Better
  7. 2024DiCoDe: Diffusion-Compressed Deep Tokens for Autoregressive Video Generation with Language Models
  8. 2025The Best of Both Worlds: Integrating Language Models and Diffusion Models for Video Generation