mlsanityData to 7 Oct 2026

PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation

arXiv 2403.04692 ↗First posted 7 Mar 2024cs.CVcited by 326

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

In this paper, we introduce PixArt-Sigma, a Diffusion Transformer model~(DiT) capable of directly generating images at 4K resolution. PixArt-Sigma represents a significant advancement over its predecessor, PixArt-α, offering images of markedly higher fidelity and improved alignment with text prompts. A key feature of PixArt-Sigma is its training efficiency. Leveraging the foundational pre-training of PixArt-α, it evolves from the `weaker' baseline to a `stronger' model via incorporating higher quality data, a process we term "weak-to-strong training". The advancements in PixArt-Sigma are twofold: (1) High-Quality Training Data: PixArt-Sigma incorporates superior-quality image data, paired with more precise and detailed image captions. (2) Efficient Token Compression: we propose a novel attention module within the DiT framework that compresses both keys and values, significantly improving efficiency and facilitating ultra-high-resolution image generation. Thanks to these improvements, PixArt-Sigma achieves superior image quality and user prompt adherence capabilities with significantly smaller model size (0.6B parameters) than existing text-to-image diffusion models, such as SDXL (2.6B parameters) and SD Cascade (5.1B parameters). Moreover, PixArt-Sigma's capability to generate 4K images supports the creation of high-resolution posters and wallpapers, efficiently bolstering the production of high-quality visual content in industries such as film and gaming.

Key facts. Cited by 326 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 43 of its 52 references that are in the map. First posted on arXiv 7 Mar 2024.

Builds on

43 of 52 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024FiT: Flexible Vision Transformer for Diffusion Model19 Feb
  2. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers16 Jan
  3. 2024PIXART-δ: Fast and Controllable Image Generation with Latent Consistency Models10 Jan
  4. 2023GenTron: Diffusion Transformers for Image and Video Generation7 Dec
  5. 2023DiffiT: Diffusion Vision Transformers for Image Generation4 Dec
  6. 2023One-step Diffusion with Distribution Matching Distillation30 Nov
  7. 2023Adversarial Diffusion Distillation28 Nov
  8. 2023ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models11 Oct
  9. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis30 Sep
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  11. 2023Fast Training of Diffusion Models with Masked Transformers15 Jun
  12. 2023DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning13 Apr
4 more
  1. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer25 Mar
  2. 2022Scalable Diffusion Models with Transformers19 Dec
  3. 2022All are Worth Words: A ViT Backbone for Diffusion Models25 Sep
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec

Other references · 27

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  6. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  8. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  9. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  10. 2020Rethinking Attention with Performerscited by 1,584
  11. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  12. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
15 more
  1. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  2. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  3. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  4. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  5. 2022Vision Transformer with Deformable Attentioncited by 293
  6. 2024Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generationcited by 249
  7. 2021Chasing Sparsity in Vision Transformers: An End-to-End Explorationcited by 181
  8. 2021SOFT: Softmax-free Transformer with Linear Complexitycited by 128
  9. 2013Stabilitycited by 119
  10. 2023DemoFusion: Democratising High-Resolution Image Generation With No cited by 105
  11. 2023SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformercited by 60
  12. 2023Wuerstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Modelscited by 50
  13. 2022A Self-Attention Ansatz for Ab-initio Quantum Chemistrycited by 45
  14. 2023CAME: Confidence-guided Adaptive Memory Efficient Optimizationcited by 34
  15. 2023Advancing Vision Transformers with Group-Mix Attentioncited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 47; 2025: 154; 2026: 125.

326 papers.

Sort:

By area:

  1. 2025Qwen-Image Technical Reportcited by 973
  2. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  3. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 331
  4. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 225
  5. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 209
  6. 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 196
  7. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  8. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 161
  9. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
  10. 2024Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiTcited by 130
  11. 2024HART: Efficient Visual Generation with Hybrid Autoregressive Transformercited by 128
  12. 2024FORA: Fast-Forward Caching in Diffusion Transformer Accelerationcited by 127
38 more
  1. 2025From Reusing to Forecasting: Accelerating Diffusion Models with TaylorSeerscited by 124
  2. 2024FasterCache: Training-Free Video Diffusion Model Acceleration with High Qualitycited by 115
  3. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 111
  4. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generationcited by 110
  5. 2025Lumina-Image 2.0: A Unified and Efficient Image Generative Frameworkcited by 93
  6. 2025LongCat-Image Technical Reportcited by 92
  7. 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformercited by 91
  8. 2025SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillationcited by 85
  9. 2024GenAI Arena: An Open Evaluation Platform for Generative Modelscited by 81
  10. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  11. 2025PixelDiT: Pixel Diffusion Transformers for Image Generationcited by 75
  12. 2024DreamLIP: Language-Image Pre-training with Long Captionscited by 72
  13. 2025TIIF-Bench: How Does Your T2I Model Follow Your Instructions?cited by 64
  14. 2024Flash Diffusion: Accelerating Any Conditional Diffusion Model for Few Steps Image Generationcited by 56
  15. 2025From Reflection to Perfection: Scaling Inference-Time Optimization for Text-to-Image Diffusion Models via Reflection Tuningcited by 53
  16. 2024From Pixels to Prose: A Large Dataset of Dense Image Captionscited by 52
  17. 2024U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformerscited by 51
  18. 2025OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generationcited by 50
  19. 2025DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generationcited by 49
  20. 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocitiescited by 46
  21. 2024Margin-aware Preference Optimization for Aligning Diffusion Models without Referencecited by 45
  22. 2025UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editingcited by 43
  23. 2024SynerGen-VL: Towards Synergistic Image Understanding and Generation with Vision Experts and Token Foldingcited by 42
  24. 2024Autoregressive Models in Vision: A Surveycited by 39
  25. 2024EvalMuse-40K: A Reliable and Fine-Grained Benchmark with Comprehensive Human Annotations for Text-to-Image Generation Model Evaluationcited by 37
  26. 2024Token Merging for Training-Free Semantic Binding in Text-to-Image Synthesiscited by 35
  27. 2025Reangle-A-Video: 4D Video Generation as Video-to-Video Translationcited by 35
  28. 2025VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Modelscited by 34
  29. 2025AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Modelscited by 33
  30. 2025Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage, but Not Direct the Play?cited by 33
  31. 2024Causal Diffusion Transformers for Generative Modelingcited by 31
  32. 2025Region-Adaptive Sampling for Diffusion Transformerscited by 31
  33. 2025FlowTok: Flowing Seamlessly Across Text and Image Tokenscited by 31
  34. 2025U-REPA: Aligning Diffusion U-Nets to ViTscited by 29
  35. 2025Shifting AI Efficiency From Model-Centric to Data-Centric Compressioncited by 29
  36. 2024STAR: Scale-wise Text-conditioned AutoRegressive image generationcited by 28
  37. 2024CLEAR: Conv-Like Linearization Revs Pre-Trained Diffusion Transformers Upcited by 26
  38. 2025TextAtlas5M: A Large-scale Dataset for Dense Text Image Generationcited by 25

50 most cited of 326 All 326 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025E-MMDiT: Revisiting Multimodal Diffusion Transformer Design for Fast Image Synthesis under Limited Resources
  2. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation
  3. 2024EC-DIT: Scaling Diffusion Transformers with Adaptive Expert-Choice Routing
  4. 2025Diffusion-4K: Ultra-High-Resolution Image Synthesis with Latent Diffusion Models
  5. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators
  6. 2025Rare Text Semantics Were Always There in Your Diffusion Transformer
  7. 2026HRDiT: Training-Free High-Resolution Image Generation with Off-the-Shelf Diffusion Transformer Models
  8. 2023PI3D: Efficient Text-to-3D Generation with Pseudo-Image Diffusion