mlsanityData to 7 Oct 2026

PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis

arXiv 2310.00426 ↗First posted 30 Sep 2023cs.CVcited by 854

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

The most advanced text-to-image (T2I) models require significant training costs (e.g., millions of GPU hours), seriously hindering the fundamental innovation for the AIGC community while increasing CO2 emissions. This paper introduces PIXART-α, a Transformer-based T2I diffusion model whose image generation quality is competitive with state-of-the-art image generators (e.g., Imagen, SDXL, and even Midjourney), reaching near-commercial application standards. Additionally, it supports high-resolution image synthesis up to 1024px resolution with low training cost, as shown in Figure 1 and 2. To achieve this goal, three core designs are proposed: (1) Training strategy decomposition: We devise three distinct training steps that separately optimize pixel dependency, text-image alignment, and image aesthetic quality; (2) Efficient T2I Transformer: We incorporate cross-attention modules into Diffusion Transformer (DiT) to inject text conditions and streamline the computation-intensive class-condition branch; (3) High-informative data: We emphasize the significance of concept density in text-image pairs and leverage a large Vision-Language model to auto-label dense pseudo-captions to assist text-image alignment learning. As a result, PIXART-α's training speed markedly surpasses existing large-scale T2I models, e.g., PIXART-α only takes 10.8% of Stable Diffusion v1.5's training time (675 vs. 6,250 A100 GPU days), saving nearly 300,000 (26,000 vs. 320,000) and reducing 90% CO2 emissions. Moreover, compared with a larger SOTA model, RAPHAEL, our training cost is merely 1%. Extensive experiments demonstrate that PIXART-α excels in image quality, artistry, and semantic control. We hope PIXART-α will provide new insights to the AIGC community and startups to accelerate building their own high-quality yet low-cost generative models from scratch.

Key facts. Cited by 854 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 61 of its 79 references that are in the map. First posted on arXiv 30 Sep 2023.

Builds on

61 of 79 references.

Closest ancestors · 14

Same direction, newest first

  1. 2023SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models10 Sep
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  3. 2023Fast Training of Diffusion Models with Masked Transformers15 Jun
  4. 2023DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning13 Apr
  5. 2022Scalable Diffusion Models with Transformers19 Dec
  6. 2022All are Worth Words: A ViT Backbone for Diffusion Models25 Sep
  7. 2022Classifier-Free Diffusion Guidance26 Jul
  8. 2022A Study on the Evaluation of Generative Models22 Jun
  9. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  10. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  11. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  12. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
2 more
  1. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  2. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2023Segment Anythingcited by 8,158
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2020End-to-End Object Detection with Transformerscited by 7,384
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
35 more
  1. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  2. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  7. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  8. 2015Variational Inference with Normalizing Flowscited by 2,161
  9. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  10. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  11. 2020Point Transformercited by 1,462
  12. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  13. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  14. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  15. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  16. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  17. 2021Video Swin Transformercited by 1,090
  18. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  19. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  20. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  21. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  22. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  23. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  24. 2021Transformer in Transformercited by 632
  25. 2015Holistically-Nested Edge Detectioncited by 590
  26. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  27. 2020TransTrack: Multiple Object Tracking with Transformercited by 370
  28. 2021DeepViT: Towards Deeper Vision Transformercited by 288
  29. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 264
  30. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
  31. 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 203
  32. 2022Understanding The Robustness in Vision Transformerscited by 144
  33. 2023RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Pathscited by 141
  34. 2022ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Expertscited by 110
  35. 2021Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformerscited by 109

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 13; 2024: 267; 2025: 349; 2026: 225.

854 papers.

Sort:

By area:

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  2. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  3. 2025Qwen-Image Technical Reportcited by 973
  4. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  5. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  6. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  7. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  8. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
  9. 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
  10. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  11. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  12. 2025VACE: All-in-One Video Creation and Editingcited by 467
38 more
  1. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
  2. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  3. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
  4. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 332
  5. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 331
  6. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
  7. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 326
  8. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  9. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 286
  10. 2024Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wildcited by 281
  11. 2025MAGI-1: Autoregressive Video Generation at Scalecited by 262
  12. 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillationcited by 258
  13. 2024Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generationcited by 249
  14. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  15. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  16. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  17. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
  18. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 209
  19. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199
  20. 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 196
  21. 2024Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformerscited by 169
  22. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  23. 2024Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Predictioncited by 163
  24. 2025Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Modelcited by 163
  25. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 162
  26. 2025Native and Compact Structured Latents for 3D Generationcited by 157
  27. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
  28. 2023Cache Me if You Can: Accelerating Diffusion Models through Block Cachingcited by 149
  29. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 144
  30. 2024Accelerating Diffusion Transformers with Token-wise Feature Cachingcited by 143
  31. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 142
  32. 2025Diffusion Adversarial Post-Training for One-Step Video Generationcited by 142
  33. 2024PIXART-δ: Fast and Controllable Image Generation with Latent Consistency Modelscited by 139
  34. 2024CFG++: Manifold-constrained Classifier Free Guidance for Diffusion Modelscited by 138
  35. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
  36. 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Renderingcited by 136
  37. 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 132
  38. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130

50 most cited of 854 All 854 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026HRDiT: Training-Free High-Resolution Image Generation with Off-the-Shelf Diffusion Transformer Models
  2. 2024Diffusion Model Compression for Image-to-Image Translation
  3. 2024FBSDiff: Plug-and-Play Frequency Band Substitution of Diffusion Features for Highly Controllable Text-Driven Image Translation
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
  5. 2023Any-Size-Diffusion: Toward Efficient Text-Driven Synthesis for Any-Size HD Images
  6. 2026Pixel-Space Diffusion Transformers
  7. 2023Text-to-image Diffusion Models in Generative AI: A Survey
  8. 2026FBSDiff++: Improved Frequency Band Substitution of Diffusion Features for Efficient and Highly Controllable Text-Driven Image-to-Image Translation