# AdaptiveLoad: Towards Efficient Video Diffusion Transformer Training

arXiv: https://arxiv.org/abs/2605.17923 · HTML: https://mlsanity.com/paper/2605.17923/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 15 of its 35 references that are in the map.
- First posted on arXiv 18 May 2026.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)
- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 15 of 15)

- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models](https://mlsanity.com/paper/2309.14509/) (2023)
- [Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goals](https://mlsanity.com/paper/2407.05996/) (2024)
- [Kling-Omni Technical Report](https://mlsanity.com/paper/2512.16776/) (2025)
- [Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training](https://mlsanity.com/paper/2504.13161/) (2025)
- [DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models](https://mlsanity.com/paper/2512.24165/) (2025)
- [Scalable Training of Mixture-of-Experts Models with Megatron Core](https://mlsanity.com/paper/2603.07685/) (2026)
- [ElasticMM: Efficient Multimodal LLMs Serving with Elastic Multimodal Parallelism](https://mlsanity.com/paper/2507.10069/) (2025)
- [RL-VLA3: A Flexible and Asynchronous Reinforcement Learning Framework for VLA Training](https://mlsanity.com/paper/2602.05765/) (2026)
- [MegaScale-Data: Scaling Dataloader for Multisource Large Foundation Model Training](https://mlsanity.com/paper/2504.09844/) (2025)
- [Making LLMs Optimize Multi-Scenario CUDA Kernels Like Experts](https://mlsanity.com/paper/2603.07169/) (2026)
- [DiM-Gesture: Co-Speech Gesture Generation with Adaptive Layer Normalization Mamba-2 framework](https://mlsanity.com/paper/2408.00370/) (2024)
- [Towards Scalable Video Anomaly Retrieval: A Synthetic Video-Text Benchmark](https://mlsanity.com/paper/2506.01466/) (2025)

## Abstract (arXiv)

In video generation models, particularly world models, training large-scale video diffusion Transformers (such as DiT and MMDiT) poses significant computational challenges due to the extreme variance in sequence lengths within mixed-mode datasets. Existing bucket-based data loading strategies typically rely on "equal token length" constraints. This approach fails to account for the quadratic complexity of self-attention mechanisms, leading to severe load imbalance and underutilization of GPU resources. This paper proposes textitAdaptiveLoad, an integrated optimization framework consisting of two core components: (1) A dual-constraint adaptive load balancing system, which eliminates long-sequence bottlenecks by simultaneously limiting memory consumption and computational load (B × Sp ≤ Mcomp); (2) A fused LayerNorm-Modulate CUDA kernel, which utilizes a D-tile coalesced reduction strategy to increase throughput and alleviate memory pressure. Experimental results on the Wan 2.1 world model demonstrate that our method reduces the computational imbalance rate from 39% to 18.9%, improves peak VRAM utilization efficiency by 22.7%, and achieves an overall training throughput increase of 27.2%.