# HierarchicalPrune: Position-Aware Compression for Large-Scale Diffusion Models

arXiv: https://arxiv.org/abs/2508.04663 · HTML: https://mlsanity.com/paper/2508.04663/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 27 of its 45 references that are in the map.
- First posted on arXiv 6 Aug 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 27 of 27)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment](https://mlsanity.com/paper/2403.05135/) (2024)
- [Emu Edit: Precise Image Editing via Recognition and Generation Tasks](https://mlsanity.com/paper/2311.10089/) (2023)
- [Human Preference Score: Better Aligning Text-to-Image Models with Human Preference](https://mlsanity.com/paper/2303.14420/) (2023)
- [Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystack](https://mlsanity.com/paper/2309.15807/) (2023)
- [SDXL-Lightning: Progressive Adversarial Diffusion Distillation](https://mlsanity.com/paper/2402.13929/) (2024)
- [DropBlock: A regularization method for convolutional networks](https://mlsanity.com/paper/1810.12890/) (2018)
- [Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models](https://mlsanity.com/paper/2410.10733/) (2024)
- [Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers](https://mlsanity.com/paper/2406.01125/) (2024)
- [PTQD: Accurate Post-Training Quantization for Diffusion Models](https://mlsanity.com/paper/2305.10657/) (2023)
- [SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillation](https://mlsanity.com/paper/2503.09641/) (2025)
- [FiT: Flexible Vision Transformer for Diffusion Model](https://mlsanity.com/paper/2402.12376/) (2024)
- [LD-Pruner: Efficient Pruning of Latent Diffusion Models using Task-Agnostic Insights](https://mlsanity.com/paper/2404.11936/) (2024)
- [LAPTOP-Diff: Layer Pruning and Normalized Distillation for Compressing Diffusion Models](https://mlsanity.com/paper/2404.11098/) (2024)
- [QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning](https://mlsanity.com/paper/2402.03666/) (2024)
- [SnapGen: Taming High-Resolution Text-to-Image Models for Mobile Devices with Efficient Architectures and Training](https://mlsanity.com/paper/2412.09619/) (2024)
- [TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edge](https://mlsanity.com/paper/2307.09988/) (2023)

## Built on by (most cited first, 8 of 8)

- [Elastic Diffusion Transformer](https://mlsanity.com/paper/2602.13993/) (2026)
- [Pluggable Pruning with Contiguous Layer Distillation for Diffusion Transformers](https://mlsanity.com/paper/2511.16156/) (2025)
- [DiffusionX: Efficient Edge-Cloud Collaborative Image Generation with Multi-Round Prompt Evolution](https://mlsanity.com/paper/2510.16326/) (2025)
- [TrajShield: Trajectory-Level Safety Mediation for Defending Text-to-Video Models Against Jailbreak Attacks](https://mlsanity.com/paper/2605.01761/) (2026)
- [NanoFLUX: Distillation-Driven Compression of Large Text-to-Image Generation Models for Mobile Devices](https://mlsanity.com/paper/2602.06879/) (2026)
- [ARIA: Adaptive Region-Based Importance Allocation for Conditional Diffusion Distillation](https://mlsanity.com/paper/2606.23898/) (2026)
- [TMP: Tree-structured Mixed-policy Pruning for Large-scale Image Generation and Editing](https://mlsanity.com/paper/2606.27089/) (2026)
- [Importance-Aware Low-Rank Distillation of Diffusion Transformers](https://mlsanity.com/paper/2609.04646/) (2026)

## Abstract (arXiv)

State-of-the-art text-to-image diffusion models (DMs) achieve remarkable quality, yet their massive parameter scale (8-11B) poses significant challenges for inferences on resource-constrained devices. In this paper, we present HierarchicalPrune, a novel compression framework grounded in a key observation: DM blocks exhibit distinct functional hierarchies, where early blocks establish semantic structures while later blocks handle texture refinements. HierarchicalPrune synergistically combines three techniques: (1) Hierarchical Position Pruning, which identifies and removes less essential later blocks based on position hierarchy; (2) Positional Weight Preservation, which systematically protects early model portions that are essential for semantic structural integrity; and (3) Sensitivity-Guided Distillation, which adjusts knowledge-transfer intensity based on our discovery of block-wise sensitivity variations. As a result, our framework brings billion-scale diffusion models into a range more suitable for on-device inference, while preserving the quality of the output images. Specifically, combined with INT4 weight quantisation, HierarchicalPrune achieves 77.5-80.4% memory footprint reduction (e.g., from 15.8 GB to 3.2 GB) and 27.9-38.0% latency reduction, measured on server and consumer grade GPUs, with the minimum drop of 2.6% in GenEval score and 7% in HPSv2 score compared to the original model. Finally, our comprehensive user study with 85 participants demonstrates that HierarchicalPrune maintains perceptual quality comparable to the original model while significantly outperforming prior works.