# Vchitect-2.0: Parallel Transformer for Scaling Up Video Diffusion Models

arXiv: https://arxiv.org/abs/2501.08453 · HTML: https://mlsanity.com/paper/2501.08453/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 28 of its 70 references that are in the map.
- First posted on arXiv 14 Jan 2025.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [VideoChat: Chat-Centric Video Understanding](https://mlsanity.com/paper/2305.06355/) (2023)
- [Training Deep Nets with Sublinear Memory Cost](https://mlsanity.com/paper/1604.06174/) (2016)
- [VideoCrafter1: Open Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2310.19512/) (2023)
- [MagicVideo: Efficient Video Generation With Latent Diffusion Models](https://mlsanity.com/paper/2211.11018/) (2022)
- [Ring Attention with Blockwise Transformers for Near-Infinite Context](https://mlsanity.com/paper/2310.01889/) (2023)
- [Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation](https://mlsanity.com/paper/2309.15818/) (2023)
- [DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models](https://mlsanity.com/paper/2309.14509/) (2023)
- [MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captions](https://mlsanity.com/paper/2407.06358/) (2024)
- [Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generation](https://mlsanity.com/paper/2304.08477/) (2023)
- [PipeDream: Fast and Efficient Pipeline Parallel DNN Training](https://mlsanity.com/paper/1806.03377/) (2018)
- [M6: A Chinese Multimodal Pretrainer](https://mlsanity.com/paper/2103.00823/) (2021)
- [PyTorch Distributed: Experiences on Accelerating Data Parallel Training](https://mlsanity.com/paper/2006.15704/) (2020)
- [VEnhancer: Generative Space-Time Enhancement for Video Generation](https://mlsanity.com/paper/2407.07667/) (2024)
- [ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training](https://mlsanity.com/paper/2104.14129/) (2021)
- [VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2411.13503/) (2024)
- [GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism](https://mlsanity.com/paper/1811.06965/) (2018)

## Built on by (most cited first, 30 of 60)

- [VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness](https://mlsanity.com/paper/2503.21755/) (2025)
- [WorldScore: A Unified Evaluation Benchmark for World Generation](https://mlsanity.com/paper/2504.00983/) (2025)
- [VSA: Faster Video Diffusion with Trainable Sparse Attention](https://mlsanity.com/paper/2505.13389/) (2025)
- [Mixture of Contexts for Long Video Generation](https://mlsanity.com/paper/2508.21058/) (2025)
- [OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generation](https://mlsanity.com/paper/2505.20292/) (2025)
- [Goku: Flow Based Video Generative Foundation Models](https://mlsanity.com/paper/2502.04896/) (2025)
- [RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Control](https://mlsanity.com/paper/2502.10059/) (2025)
- [CFG-Zero*: Improved Classifier-Free Guidance for Flow Matching Models](https://mlsanity.com/paper/2503.18886/) (2025)
- [Omni-Video: Democratizing Unified Video Understanding and Generation](https://mlsanity.com/paper/2507.06119/) (2025)
- [VideoScore2: Think before You Score in Generative Video Evaluation](https://mlsanity.com/paper/2509.22799/) (2025)
- [Spatia: Video Generation with Updatable Spatial Memory](https://mlsanity.com/paper/2512.15716/) (2025)
- [4DNeX: Feed-Forward 4D Generative Modeling Made Easy](https://mlsanity.com/paper/2508.13154/) (2025)
- [DreamRelation: Relation-Centric Video Customization](https://mlsanity.com/paper/2503.07602/) (2025)
- [WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World](https://mlsanity.com/paper/2512.10958/) (2025)
- [PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generation](https://mlsanity.com/paper/2512.24551/) (2025)
- [LongVie: Multimodal-Guided Controllable Ultra-Long Video Generation](https://mlsanity.com/paper/2508.03694/) (2025)
- [Bidirectional Sparse Attention for Faster Video Diffusion Training](https://mlsanity.com/paper/2509.01085/) (2025)
- [UniVid: The Open-Source Unified Video Model](https://mlsanity.com/paper/2509.24200/) (2025)
- [Omni-Video 2: Scaling MLLM-Conditioned Diffusion for Unified Video Generation and Editing](https://mlsanity.com/paper/2602.08820/) (2026)
- [Latent Spatial Memory for Video World Models](https://mlsanity.com/paper/2606.09828/) (2026)
- [Think Before You Diffuse: Infusing Physical Rules into Video Diffusion](https://mlsanity.com/paper/2505.21653/) (2025)
- [Simulating the Visual World with Artificial Intelligence: A Roadmap](https://mlsanity.com/paper/2511.08585/) (2025)
- [Demystifying Video Reasoning](https://mlsanity.com/paper/2603.16870/) (2026)
- [Turbo2K: Towards Ultra-Efficient and High-Quality 2K Video Synthesis](https://mlsanity.com/paper/2504.14470/) (2025)
- [Enhancing Physical Plausibility in Video Generation by Reasoning the Implausibility](https://mlsanity.com/paper/2509.24702/) (2025)
- [M4V: Multimodal Mamba for Efficient Text-to-Video Generation](https://mlsanity.com/paper/2506.10915/) (2025)
- [Uniform Discrete Diffusion with Metric Path for Video Generation](https://mlsanity.com/paper/2510.24717/) (2025)
- [Light-X: Generative 4D Video Rendering with Camera and Illumination Control](https://mlsanity.com/paper/2512.05115/) (2025)
- [Mode Seeking meets Mean Seeking for Fast Long Video Generation](https://mlsanity.com/paper/2602.24289/) (2026)
- [MVU-Eval: Towards Multi-Video Understanding Evaluation for Multimodal LLMs](https://mlsanity.com/paper/2511.07250/) (2025)

## Abstract (arXiv)

We present Vchitect-2.0, a parallel transformer architecture designed to scale up video diffusion models for large-scale text-to-video generation. The overall Vchitect-2.0 system has several key designs. (1) By introducing a novel Multimodal Diffusion Block, our approach achieves consistent alignment between text descriptions and generated video frames, while maintaining temporal coherence across sequences. (2) To overcome memory and computational bottlenecks, we propose a Memory-efficient Training framework that incorporates hybrid parallelism and other memory reduction techniques, enabling efficient training of long video sequences on distributed systems. (3) Additionally, our enhanced data processing pipeline ensures the creation of Vchitect T2V DataVerse, a high-quality million-scale training dataset through rigorous annotation and aesthetic evaluation. Extensive benchmarking demonstrates that Vchitect-2.0 outperforms existing methods in video quality, training efficiency, and scalability, serving as a suitable base for high-fidelity video generation.