mlsanityData to 14 Oct 2026

FastFLUX: Pruning FLUX with Block-wise Replacement and Sandwich Training

arXiv 2506.10035 ↗First posted 10 Jun 2025cs.GR cs.AIcited by 5

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Recent advancements in text-to-image (T2I) generation have led to the emergence of highly expressive models such as diffusion transformers (DiTs), exemplified by FLUX. However, their massive parameter sizes lead to slow inference, high memory usage, and poor deployability. Existing acceleration methods (e.g., single-step distillation and attention pruning) often suffer from significant performance degradation and incur substantial training costs. To address these limitations, we propose FastFLUX, an architecture-level pruning framework designed to enhance the inference efficiency of FLUX. At its core is the Block-wise Replacement with Linear Layers (BRLL) method, which replaces structurally complex residual branches in ResBlocks with lightweight linear layers while preserving the original shortcut connections for stability. Furthermore, we introduce Sandwich Training (ST), a localized fine-tuning strategy that leverages LoRA to supervise neighboring blocks, mitigating performance drops caused by structural replacement. Experiments show that our FastFLUX maintains high image quality under both qualitative and quantitative evaluations, while significantly improving inference speed, even with 20% of the hierarchy pruned. Our code will be available soon.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 63 of its 72 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

63 of 72 references.

Closest ancestors · 18

Same direction, newest first

  1. 2024Efficient Fine-Tuning and Concept Suppression for Pruned Diffusion Models19 Dec
  2. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models7 Nov
  3. 2024Imagine Flash: Accelerating Emu Diffusion Models with Backward Distillation8 May
  4. 2024LD-Pruner: Efficient Pruning of Latent Diffusion Models using Task-Agnostic Insights18 Apr
  5. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
  6. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models5 Oct
  7. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis30 Sep
  8. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  9. 2023Temporal Dynamic Quantization for Diffusion Models4 Jun
  10. 2023Structural Pruning for Diffusion Models18 May
  11. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  12. 2022Scalable Diffusion Models with Transformers19 Dec
6 more
  1. 2022Post-training Quantization on Diffusion Models28 Nov
  2. 2022Accelerating Diffusion Sampling with Classifier-based Feature Distillation22 Nov
  3. 2022On Distillation of Guided Diffusion Models6 Oct
  4. 2022A Survey on Generative Diffusion Model6 Sep
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  6. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 45

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  5. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  10. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  11. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  12. 2020Rethinking Attention with Performerscited by 1,601
33 more
  1. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  2. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  3. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  4. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  5. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  6. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  7. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  8. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  9. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  10. 2019Are Sixteen Heads Really Better than One?cited by 816
  11. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  12. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 702
  13. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 700
  14. 2018Rethinking the Value of Network Pruningcited by 599
  15. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
  16. 2016Learning Structured Sparsity in Deep Neural Networkscited by 541
  17. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 439
  18. 2021Post-Training Quantization for Vision Transformercited by 278
  19. 2019Depth-Adaptive Transformercited by 236
  20. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 166
  21. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 151
  22. 2024Stable Flow: Vital Layers for Training-Free Image Editingcited by 113
  23. 2025SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillationcited by 87
  24. 2023X-Pruner: eXplainable Pruning for Vision Transformerscited by 68
  25. 2023Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformerscited by 67
  26. 2025One Diffusion Step to Real-World Super-Resolution via Flow Trajectory Distillationcited by 41
  27. 2024Attention Calibration for Disentangled Text-to-Image Personalizationcited by 31
  28. 2024Getting it Right: Improving Spatial Consistency in Text-to-Image Modelscited by 30
  29. 2022DropNet: Reducing Neural Network Complexity via Iterative Pruningcited by 29
  30. 2020Pruning via Iterative Ranking of Sensitivity Statisticscited by 26
  31. 2024Depth Prompting for Sensor-Agnostic Depth Estimationcited by 26
  32. 2024Iterative Filter Pruning for Concatenation-based CNN Architecturescited by 4
  33. 2025Revisiting Large Language Model Pruning using Neuron Semantic Attributioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Pluggable Pruning with Contiguous Layer Distillation for Diffusion Transformerscited by 7
  2. 2025Fast3Dcache: Training-free 3D Geometry Synthesis Accelerationcited by 3
  3. 2026Cross-Space Distillation: Teaching One-Step Students with Modern Diffusion Teacherscited by 2
  4. 2026TMP: Tree-structured Mixed-policy Pruning for Large-scale Image Generation and Editingcited by 0
  5. 2026Importance-Aware Low-Rank Distillation of Diffusion Transformerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Token Pruning for In-Context Generation in Diffusion Transformers
  2. 2024Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiT
  3. 2024TinyFusion: Diffusion Transformers Learned Shallow
  4. 2026Diff-ES: Stage-wise Structural Diffusion Pruning via Evolutionary Search
  5. 2024Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers
  6. 2024Not All Prompts Are Made Equal: Prompt-based Pruning of Text-to-Image Diffusion Models
  7. 2026DiffSparse: Accelerating Diffusion Transformers with Learned Token Sparsity
  8. 2025Dense2MoE: Restructuring Diffusion Transformer to MoE for Efficient Text-to-Image Generation