# EDiT: Efficient Diffusion Transformers with Linear Compressed Attention

arXiv: https://arxiv.org/abs/2503.16726 · HTML: https://mlsanity.com/paper/2503.16726/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 21 of its 26 references that are in the map.
- First posted on arXiv 20 Mar 2025.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers](https://mlsanity.com/paper/2410.10629/) (2024)
- [PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation](https://mlsanity.com/paper/2403.04692/) (2024)
- [Neighborhood Attention Transformer](https://mlsanity.com/paper/2204.07143/) (2022)
- [SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer](https://mlsanity.com/paper/2501.18427/) (2025)
- [Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiT](https://mlsanity.com/paper/2406.18583/) (2024)
- [ZigMa: A DiT-style Zigzag Mamba Diffusion Model](https://mlsanity.com/paper/2403.13802/) (2024)
- [LinFusion: 1 GPU, 1 Minute, 16K Image](https://mlsanity.com/paper/2409.02097/) (2024)
- [Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budget](https://mlsanity.com/paper/2407.15811/) (2024)
- [Dimba: Transformer-Mamba Diffusion Models](https://mlsanity.com/paper/2406.01159/) (2024)
- [KOALA: Empirical Lessons Toward Memory-Efficient and Fast Diffusion Models for Text-to-Image Synthesis](https://mlsanity.com/paper/2312.04005/) (2023)

## Built on by (most cited first, 8 of 8)

- [DrDiff: Dynamic Routing Diffusion with Hierarchical Attention for Breaking the Efficiency-Quality Trade-off](https://mlsanity.com/paper/2509.02785/) (2025)
- [Exploring Diffusion Transformer Designs via Grafting](https://mlsanity.com/paper/2506.05340/) (2025)
- [FSVideo: Fast Speed Video Diffusion Model in a Highly-Compressed Latent Space](https://mlsanity.com/paper/2602.02092/) (2026)
- [Dynamic Differential Linear Attention: Enhancing Linear Diffusion Transformer for High-Quality Image Generation](https://mlsanity.com/paper/2601.13683/) (2026)
- [NanoFLUX: Distillation-Driven Compression of Large Text-to-Image Generation Models for Mobile Devices](https://mlsanity.com/paper/2602.06879/) (2026)
- [Contact-Guided 3D Genome Structure Generation of E. coli via Diffusion Transformers](https://mlsanity.com/paper/2603.07472/) (2026)
- [Beyond Attention Masks: Instruction Anchoring for Efficient In-Context Diffusion Generation](https://mlsanity.com/paper/2608.21229/) (2026)
- [LinMorph: Learning to Morph Weights for Efficient Video DiT Linearization](https://mlsanity.com/paper/2610.17914/) (2026)

## Abstract (arXiv)

Diffusion Transformers (DiTs) have emerged as a leading architecture for text-to-image synthesis, producing high-quality and photorealistic images. However, the quadratic scaling properties of the attention in DiTs hinder image generation with higher resolution or on devices with limited resources. This work introduces an efficient diffusion transformer (EDiT) to alleviate these efficiency bottlenecks in conventional DiTs and Multimodal DiTs (MM-DiTs). First, we present a novel linear compressed attention method that uses a multi-layer convolutional network to modulate queries with local information while keys and values are aggregated spatially. Second, we formulate a hybrid attention scheme for multimodal inputs that combines linear attention for image-to-image interactions and standard scaled dot-product attention for interactions involving prompts. Merging these two approaches leads to an expressive, linear-time Multimodal Efficient Diffusion Transformer (MM-EDiT). We demonstrate the effectiveness of the EDiT and MM-EDiT architectures by integrating them into PixArt-Sigma (conventional DiT) and Stable Diffusion 3.5-Medium (MM-DiT), achieving up to 2.2x speedup with comparable image quality after distillation.