# xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism

arXiv: https://arxiv.org/abs/2411.01738 · HTML: https://mlsanity.com/paper/2411.01738/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 31 of its 44 references that are in the map.
- First posted on arXiv 4 Nov 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2305.01569/) (2023)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [Movie Gen: A Cast of Media Foundation Models](https://mlsanity.com/paper/2410.13720/) (2024)
- [Latte: Latent Diffusion Transformer for Video Generation](https://mlsanity.com/paper/2401.03048/) (2024)
- [All are Worth Words: A ViT Backbone for Diffusion Models](https://mlsanity.com/paper/2209.12152/) (2022)
- [Ring Attention with Blockwise Transformers for Near-Infinite Context](https://mlsanity.com/paper/2310.01889/) (2023)
- [DeepCache: Accelerating Diffusion Models for Free](https://mlsanity.com/paper/2312.00858/) (2023)
- [Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding](https://mlsanity.com/paper/2405.08748/) (2024)
- [DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models](https://mlsanity.com/paper/2309.14509/) (2023)
- [Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers](https://mlsanity.com/paper/2406.01125/) (2024)
- [Real-Time Video Generation with Pyramid Attention Broadcast](https://mlsanity.com/paper/2408.12588/) (2024)
- [Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching](https://mlsanity.com/paper/2406.01733/) (2024)
- [DiTFastAttn: Attention Compression for Diffusion Transformer Models](https://mlsanity.com/paper/2406.08552/) (2024)
- [DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models](https://mlsanity.com/paper/2402.19481/) (2024)
- [Scaling Diffusion Transformers to 16 Billion Parameters](https://mlsanity.com/paper/2407.11633/) (2024)
- [USP: A Unified Sequence Parallelism Approach for Long Context Generative AI](https://mlsanity.com/paper/2405.07719/) (2024)
- [Scaling Laws For Diffusion Transformers](https://mlsanity.com/paper/2410.08184/) (2024)
- [A Study on the Evaluation of Generative Models](https://mlsanity.com/paper/2206.10935/) (2022)
- [PipeFusion: Patch-level Pipeline Parallelism for Diffusion Transformers Inference](https://mlsanity.com/paper/2405.14430/) (2024)

## Built on by (most cited first, 30 of 56)

- [Radial Attention: O(nlog n) Sparse Attention with Energy Decay for Long Video Generation](https://mlsanity.com/paper/2506.19852/) (2025)
- [Infinite Worlds with Versatile Interactions](https://mlsanity.com/paper/2607.07534/) (2026)
- [Efficient Diffusion Models: A Survey](https://mlsanity.com/paper/2502.06805/) (2025)
- [Training-Free Efficient Video Generation via Dynamic Token Carving](https://mlsanity.com/paper/2505.16864/) (2025)
- [Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficient](https://mlsanity.com/paper/2411.17787/) (2024)
- [StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generation](https://mlsanity.com/paper/2511.07399/) (2025)
- [Phased DMD: Few-step Distribution Matching Distillation via Score Matching within Subintervals](https://mlsanity.com/paper/2510.27684/) (2025)
- [QVGen: Pushing the Limit of Quantized Video Generative Models](https://mlsanity.com/paper/2505.11497/) (2025)
- [Efficient and Adaptable Overlapping for Computation and Communication via Signaling and Reordering](https://mlsanity.com/paper/2504.19519/) (2025)
- [LeMiCa: Lexicographic Minimax Path Caching for Efficient Diffusion-Based Video Generation](https://mlsanity.com/paper/2511.00090/) (2025)
- [MeanCache: From Instantaneous to Average Velocity for Accelerating Flow Matching Inference](https://mlsanity.com/paper/2601.19961/) (2026)
- [Where Do the Joules Go? Diagnosing Inference Energy Consumption](https://mlsanity.com/paper/2601.22076/) (2026)
- [SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routing](https://mlsanity.com/paper/2603.08982/) (2026)
- [Inferix: A Block-Diffusion based Next-Generation Inference Engine for World Simulation](https://mlsanity.com/paper/2511.20714/) (2025)
- [SRDiffusion: Accelerate Video Diffusion Inference via Sketching-Rendering Cooperation](https://mlsanity.com/paper/2505.19151/) (2025)
- [Efficient Video Diffusion Models: Advancements and Challenges](https://mlsanity.com/paper/2604.15911/) (2026)
- [Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment](https://mlsanity.com/paper/2604.24447/) (2026)
- [SuperGen: An Efficient Ultra-high-resolution Video Generation System with Sketching and Tiling](https://mlsanity.com/paper/2508.17756/) (2025)
- [An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Models](https://mlsanity.com/paper/2608.16887/) (2026)
- [FlexCache: Flexible Approximate Cache System for Video Diffusion](https://mlsanity.com/paper/2501.04012/) (2024)
- [Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms](https://mlsanity.com/paper/2603.28489/) (2026)
- [LazyDrag: Enabling Stable Drag-Based Editing on Multi-Modal Diffusion Transformers via Explicit Correspondence](https://mlsanity.com/paper/2509.12203/) (2025)
- [TetriServe: Efficiently Serving Mixed DiT Workloads](https://mlsanity.com/paper/2510.01565/) (2025)
- [Cornfigurator: Automated Planning for Any-to-Any Multimodal Model Serving](https://mlsanity.com/paper/2512.14098/) (2025)
- [From Noise to Nuance: Advances in Deep Generative Image Models](https://mlsanity.com/paper/2412.09656/) (2024)
- [db-SP: Accelerating Sparse Attention for Visual Generative Models with Dual-Balanced Sequence Parallelism](https://mlsanity.com/paper/2511.23113/) (2025)
- [M*: A Modular, Extensible, Serving System for Multimodal Models](https://mlsanity.com/paper/2606.12688/) (2026)
- [Communication-Efficient Diffusion Denoising Parallelization via Reuse-then-Predict Mechanism](https://mlsanity.com/paper/2505.14741/) (2025)
- [Accelerating Parallel Diffusion Model Serving with Residual Compression](https://mlsanity.com/paper/2507.17511/) (2025)
- [Jano: Adaptive Diffusion Generation with Early-stage Convergence Awareness](https://mlsanity.com/paper/2603.00519/) (2026)

## Abstract (arXiv)

Diffusion models are pivotal for generating high-quality images and videos. Inspired by the success of OpenAI's Sora, the backbone of diffusion models is evolving from U-Net to Transformer, known as Diffusion Transformers (DiTs). However, generating high-quality content necessitates longer sequence lengths, exponentially increasing the computation required for the attention mechanism, and escalating DiTs inference latency. Parallel inference is essential for real-time DiTs deployments, but relying on a single parallel method is impractical due to poor scalability at large scales. This paper introduces xDiT, a comprehensive parallel inference engine for DiTs. After thoroughly investigating existing DiTs parallel approaches, xDiT chooses Sequence Parallel (SP) and PipeFusion, a novel Patch-level Pipeline Parallel method, as intra-image parallel strategies, alongside CFG parallel for inter-image parallelism. xDiT can flexibly combine these parallel approaches in a hybrid manner, offering a robust and scalable solution. Experimental results on two 8xL40 GPUs (PCIe) nodes interconnected by Ethernet and an 8xA100 (NVLink) node showcase xDiT's exceptional scalability across five state-of-the-art DiTs. Notably, we are the first to demonstrate DiTs scalability on Ethernet-connected GPU clusters. xDiT is available at https://github.com/xdit-project/xDiT.