mlsanityData to 7 Oct 2026

EC-DIT: Scaling Diffusion Transformers with Adaptive Expert-Choice Routing

arXiv 2410.02098 ↗First posted 2 Oct 2024cs.CV cs.LGcited by 29

On the map: Diffusion models → Diffusion foundations and sampling

also in 1 more
Abstract (arXiv)

Diffusion transformers have been widely adopted for text-to-image synthesis. While scaling these models up to billions of parameters shows promise, the effectiveness of scaling beyond current sizes remains underexplored and challenging. By explicitly exploiting the computational heterogeneity of image generations, we develop a new family of Mixture-of-Experts (MoE) models (EC-DIT) for diffusion transformers with expert-choice routing. EC-DIT learns to adaptively optimize the compute allocated to understand the input texts and generate the respective image patches, enabling heterogeneous computation aligned with varying text-image complexities. This heterogeneity provides an efficient way of scaling EC-DIT up to 97 billion parameters and achieving significant improvements in training convergence, text-to-image alignment, and overall generation quality over dense models and conventional MoE models. Through extensive ablations, we show that EC-DIT demonstrates superior scalability and adaptive compute allocation by recognizing varying textual importance through end-to-end training. Notably, in text-to-image alignment evaluation, our largest models achieve a state-of-the-art GenEval score of 71.68% and still maintain competitive inference speed with intuitive interpretability.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 37 of its 46 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

37 of 46 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024Scaling Diffusion Transformers to 16 Billion Parameters16 Jul
  2. 2024Switch Diffusion Transformer: Synergizing Denoising Tasks with Sparse Mixture-of-Experts14 Mar
  3. 2023Denoising Task Routing for Diffusion Models11 Oct
  4. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis30 Sep
  5. 2023Fast Training of Diffusion Models with Masked Transformers15 Jun
  6. 2023Multi-Architecture Multi-Expert Diffusion Models8 Jun
  7. 2022Scalable Diffusion Models with Transformers19 Dec
  8. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  9. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 28

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2022Flow Matching for Generative Modelingcited by 5,426
  6. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  7. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  8. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  9. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  10. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  11. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  12. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
16 more
  1. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  2. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  3. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  4. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  5. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  6. 2022ST-MoE: Designing Stable and Transferable Sparse Expert Modelscited by 391
  7. 2023From Sparse to Soft Mixtures of Expertscited by 245
  8. 2024MM1: Methods, Analysis & Insights from Multimodal LLM Pre-trainingcited by 230
  9. 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Modelscited by 172
  10. 2023Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generationcited by 159
  11. 2023RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Pathscited by 141
  12. 2022ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Expertscited by 110
  13. 2020Improving Image Captioning with Better Use of Captionscited by 84
  14. 2023Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inferencecited by 69
  15. 2023VeCLIP: Improving CLIP Training via Visual-enriched Captionscited by 68
  16. 2024MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Expertscited by 61

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 14; 2026: 14.

29 papers.

Sort:

By area:

  1. 2025DiffMoE: Dynamic Token Selection for Scalable Diffusion Transformerscited by 23
  2. 2025Routing Matters in MoE: Scaling Diffusion Transformers with Explicit Routing Guidancecited by 23
  3. 2025TREAD: Token Routing for Efficient Architecture-agnostic Diffusion Trainingcited by 21
  4. 2024Scaling Properties of Diffusion Models for Perceptual Taskscited by 15
  5. 2025Instruction-Following Pruning for Large Language Modelscited by 14
  6. 2025Expert Race: A Flexible Routing Strategy for Scaling Diffusion Transformer with Mixture of Expertscited by 13
  7. 2025Scale-wise Distillation of Diffusion Modelscited by 11
  8. 2025TimeStep Master: Asymmetrical Mixture of Timestep LoRA Experts for Versatile and Efficient Diffusion Models in Visioncited by 9
  9. 2025ERMoE: Eigen-Reparameterized Mixture-of-Experts for Stable Routing and Interpretable Specializationcited by 6
  10. 2025Efficient Training of Diffusion Mixture-of-Experts Models: A Practical Recipecited by 6
  11. 2025Route Experts by Sequence, not by Tokencited by 5
  12. 2025ALTER: All-in-One Layer Pruning and Temporal Expert Routing for Efficient Diffusion Generationcited by 4
17 more
  1. 2026Expert Threshold Routing for Autoregressive Language Modeling with Dynamic Computation Allocation and Load Balancingcited by 3
  2. 2026Expert-Choice Routing Enables Adaptive Computation in Diffusion Language Modelscited by 3
  3. 2025LaTtE-Flow: Layerwise Timestep-Expert Flow-based Transformercited by 1
  4. 2025InterMoE: Individual-Specific 3D Human Interaction Generation via Dynamic Temporal-Selective MoEcited by 1
  5. 2026Expert-Data Alignment Governs Generation Quality in Decentralized Diffusion Modelscited by 1
  6. 2026Complete-muE: Optimal Hyperparameter Transfer and Scaling for MoE Modelscited by 1
  7. 2025InstructMoLE: Instruction-Guided Mixture of Low-rank Experts for Multi-Conditional Image Generationcited by 0
  8. 2026From Agnostic to Specific: Latent Preference Diffusion for Multi-Behavior Sequential Recommendationcited by 0
  9. 2026CARE-Edit: Condition-Aware Routing of Experts for Contextual Image Editingcited by 0
  10. 2026FocusDiT: Masking Queries in Diffusion Transformers for Fine-grained Image Generationcited by 0
  11. 2026MoECa: Aligning Feature Reuse with Expert Decomposition in Diffusion Transformerscited by 0
  12. 2026MoECodec: Image Compression for joint human and machine perception via Mixture-of-Expertscited by 0
  13. 2026Focusing on What Matters: Saliency-Harnessing Accurate Routing for Diffusion MoEcited by 0
  14. 2026MMOE: Modernizing Diffusion Transformers with Efficient Expert Designcited by 0
  15. 2026SonicWeave: Chunk-Routed Mixture-of-Experts for Unified Audio Scene Generationcited by 0
  16. 2026SAGE: Subspace Alignment for Classifier-Free Guidance in Mixture-of-Experts Diffusion Modelscited by 0
  17. 2026ITC-MoE: Importance-guided Token-aware Compression for MoE Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Dense2MoE: Restructuring Diffusion Transformer to MoE for Efficient Text-to-Image Generation
  2. 2024Efficient Scaling of Diffusion Transformers for Text-to-Image Generation
  3. 2026Abra: Scaling Diffusion Image Training
  4. 2025DiT-Air: Revisiting the Efficiency of Diffusion Model Architecture Design in Text to Image Generation
  5. 2026Nucleus-Image: Sparse MoE for Image Generation
  6. 2025Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion Transformers via In-Context Reflection
  7. 2024On the Scalability of Diffusion-based Text-to-Image Generation
  8. 2022Exploring Transformer Backbones for Image Diffusion Models