# Qihoo-T2X: An Efficient Proxy-Tokenized Diffusion Transformer for Text-to-Any-Task

arXiv: https://arxiv.org/abs/2409.04005 · HTML: https://mlsanity.com/paper/2409.04005/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 6 Sep 2024.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 15 of 15)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [Efficiently Scaling Transformer Inference](https://mlsanity.com/paper/2211.05102/) (2022)
- [PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation](https://mlsanity.com/paper/2403.04692/) (2024)
- [Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding](https://mlsanity.com/paper/2405.08748/) (2024)
- [PIXART-δ: Fast and Controllable Image Generation with Latent Consistency Models](https://mlsanity.com/paper/2401.05252/) (2024)
- [Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformers](https://mlsanity.com/paper/2405.05945/) (2024)
- [Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiT](https://mlsanity.com/paper/2406.18583/) (2024)
- [Agent Attention: On the Integration of Softmax and Linear Attention](https://mlsanity.com/paper/2312.08874/) (2023)
- [EasyAnimate: High-Performance Video Generation Framework with Hybrid Windows Attention and Reward Backpropagation](https://mlsanity.com/paper/2405.18991/) (2024)
- [Efficient Diffusion Transformer with Step-wise Dynamic Attention Mediators](https://mlsanity.com/paper/2408.05710/) (2024)

## Built on by (most cited first, 9 of 9)

- [Training-free and Adaptive Sparse Attention for Efficient Long Video Generation](https://mlsanity.com/paper/2502.21079/) (2025)
- [RelaCtrl: Relevance-Guided Efficient Control for Diffusion Transformers](https://mlsanity.com/paper/2502.14377/) (2025)
- [Scale-wise Distillation of Diffusion Models](https://mlsanity.com/paper/2503.16397/) (2025)
- [InstaDrive: Instance-Aware Driving World Models for Realistic and Consistent Video Generation](https://mlsanity.com/paper/2602.03242/) (2026)
- [Efficient Video Diffusion Models: Advancements and Challenges](https://mlsanity.com/paper/2604.15911/) (2026)
- [MoFu: Scale-Aware Modulation and Fourier Fusion for Multi-Subject Video Generation](https://mlsanity.com/paper/2512.22310/) (2025)
- [NAMI: Efficient Image Generation via Bridged Progressive Rectified Flow Transformers](https://mlsanity.com/paper/2503.09242/) (2025)
- [ConsisDrive: Identity-Preserving Driving World Models for Video Generation by Instance Mask](https://mlsanity.com/paper/2602.03213/) (2026)
- [FedCVU: Federated Learning for Cross-View Video Understanding](https://mlsanity.com/paper/2603.21647/) (2026)

## Abstract (arXiv)

The global self-attention mechanism in diffusion transformers involves redundant computation due to the sparse and redundant nature of visual information, and the attention map of tokens within a spatial window shows significant similarity. To address this redundancy, we propose the Proxy-Tokenized Diffusion Transformer (PT-DiT), which employs sparse representative token attention (where the number of representative tokens is much smaller than the total number of tokens) to model global visual information efficiently. Specifically, within each transformer block, we compute an averaging token from each spatial-temporal window to serve as a proxy token for that region. The global semantics are captured through the self-attention of these proxy tokens and then injected into all latent tokens via cross-attention. Simultaneously, we introduce window and shift window attention to address the limitations in detail modeling caused by the sparse attention mechanism. Building on the well-designed PT-DiT, we further develop the Qihoo-T2X family, which includes a variety of models for T2I, T2V, and T2MV tasks. Experimental results show that PT-DiT achieves competitive performance while reducing the computational complexity in both image and video generation tasks (e.g., a 49% reduction compared to DiT and a 34% reduction compared to PixArt-α). The visual exhibition and source code of Qihoo-T2X is available at https://360cvgroup.github.io/Qihoo-T2X/.