mlsanityData to 14 Oct 2026

SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers

arXiv 2410.10629 ↗First posted 14 Oct 2024cs.CVcited by 340

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We introduce Sana, a text-to-image framework that can efficiently generate images up to 4096×4096 resolution. Sana can synthesize high-resolution, high-quality images with strong text-image alignment at a remarkably fast speed, deployable on laptop GPU. Core designs include: (1) Deep compression autoencoder: unlike traditional AEs, which compress images only 8×, we trained an AE that can compress images 32×, effectively reducing the number of latent tokens. (2) Linear DiT: we replace all vanilla attention in DiT with linear attention, which is more efficient at high resolutions without sacrificing quality. (3) Decoder-only text encoder: we replaced T5 with modern decoder-only small LLM as the text encoder and designed complex human instruction with in-context learning to enhance the image-text alignment. (4) Efficient training and sampling: we propose Flow-DPM-Solver to reduce sampling steps, with efficient caption labeling and selection to accelerate convergence. As a result, Sana-0.6B is very competitive with modern giant diffusion model (e.g. Flux-12B), being 20 times smaller and 100+ times faster in measured throughput. Moreover, Sana-0.6B can be deployed on a 16GB laptop GPU, taking less than 1 second to generate a 1024×1024 resolution image. Sana enables content creation at low cost. Code and model will be publicly released.

Key facts. Cited by 340 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 50 of its 61 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

50 of 61 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiT5 Jun
  2. 2024U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers4 May
  3. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation7 Mar
  4. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
  5. 2023Diffusion Models Without Attention30 Nov
  6. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis30 Sep
  7. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  8. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  9. 2022Scalable Diffusion Models with Transformers19 Dec
  10. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  11. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  12. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
2 more
  1. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  2. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  4. 2022Flow Matching for Generative Modelingcited by 5,702
  5. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  7. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  8. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  9. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
  10. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  11. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  12. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
24 more
  1. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  2. 2016Language Modeling with Gated Convolutional Networkscited by 702
  3. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  4. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  5. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  6. 2018Efficient Attention: Attention with Linear Complexitiescited by 368
  7. 2023The Impact of Positional Encoding on Length Generalization in Transformerscited by 327
  8. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 298
  9. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 279
  10. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 269
  11. 2024Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generationcited by 256
  12. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 243
  13. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
  14. 2020How Much Position Information Do Convolutional Neural Networks Encode?cited by 172
  15. 2022Transformer Language Models without Positional Encodings Still Learn Positional Informationcited by 150
  16. 2024Playground v3: Improving Text-to-Image Alignment with Deep-Fusion Large Language Modelscited by 116
  17. 2024DiM: Diffusion Mamba for Efficient High-Resolution Image Synthesiscited by 72
  18. 2024UltraPixel: Advancing Ultra-High-Resolution Image Synthesis to New Peakscited by 65
  19. 2024DiG: Scalable and Efficient Diffusion Models with Gated Linear Attentioncited by 56
  20. 2022SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximationcited by 55
  21. 2022Hydra Attention: Efficient Attention with Many Headscited by 55
  22. 2024Exploring the Role of Large Language Models in Prompt Encoding for Diffusion Modelscited by 45
  23. 2024LLM4GEN: Leveraging Semantic Representation of LLMs for Text-to-Image Generationcited by 30
  24. 2024Condition-Aware Neural Network for Controlled Image Generationcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 19; 2025: 161; 2026: 160.

340 papers.

Sort:

By area:

  1. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 393
  2. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
  3. 2025Transfer between Modalities with MetaQueriescited by 236
  4. 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 236
  5. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsitycited by 217
  6. 2024Taming Rectified Flow for Inversion and Editingcited by 204
  7. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 170
  8. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  9. 2025Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200kcited by 163
  10. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 143
  11. 2025Video World Models with Long-term Spatial Memorycited by 136
  12. 2025Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attentioncited by 110
38 more
  1. 2025Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutationcited by 106
  2. 2025TempFlow-GRPO: When Timing Matters for GRPO in Flow Modelscited by 97
  3. 2025Lumina-Image 2.0: A Unified and Efficient Image Generative Frameworkcited by 96
  4. 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 90
  5. 2025Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understandingcited by 88
  6. 2025SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillationcited by 87
  7. 2025Improving the Diffusability of Autoencoderscited by 80
  8. 2025TIIF-Bench: How Does Your T2I Model Follow Your Instructions?cited by 69
  9. 2025Radial Attention: O(nlog n) Sparse Attention with Energy Decay for Long Video Generationcited by 69
  10. 2025Diffusion-4K: Ultra-High-Resolution Image Synthesis with Latent Diffusion Modelscited by 65
  11. 2025Long-Context State-Space Video World Modelscited by 64
  12. 2025SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attentioncited by 58
  13. 2025From Reflection to Perfection: Scaling Inference-Time Optimization for Text-to-Image Diffusion Models via Reflection Tuningcited by 56
  14. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 55
  15. 2025DiT4SR: Taming Diffusion Transformer for Real-World Image Super-Resolutioncited by 53
  16. 2025GoT-R1: Unleashing Reasoning Capability of MLLM for Visual Generation with Reinforcement Learningcited by 52
  17. 2024Improved Video VAE for Latent Video Diffusion Modelcited by 48
  18. 2025DiP: Taming Diffusion Models in Pixel Spacecited by 48
  19. 2025DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Spacecited by 47
  20. 2025OpenUni: A Simple Baseline for Unified Multimodal Understanding and Generationcited by 46
  21. 2024TinyFusion: Diffusion Transformers Learned Shallowcited by 45
  22. 2025EEdit: Rethinking the Spatial and Temporal Redundancy for Efficient Image Editingcited by 45
  23. 2025InfinityStar: Unified Spacetime AutoRegressive Modeling for Visual Generationcited by 45
  24. 2025UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editingcited by 44
  25. 2025Follow-Your-Emoji-Faster: Towards Efficient, Fine-Controllable, and Expressive Freestyle Portrait Animationcited by 43
  26. 2025Skywork UniPic 2.0: Building Kontext Model with Online RL for Unified Multimodal Modelcited by 41
  27. 2024Region-Aware Text-to-Image Generation via Hard Binding and Soft Refinementcited by 37
  28. 2025REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Trainingcited by 37
  29. 2025Lumina-mGPT 2.0: Stand-Alone AutoRegressive Image Modelingcited by 37
  30. 2025Training-Free Efficient Video Generation via Dynamic Token Carvingcited by 34
  31. 2025Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformerscited by 34
  32. 2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Modelscited by 34
  33. 2025StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generationcited by 30
  34. 2024CLEAR: Conv-Like Linearization Revs Pre-Trained Diffusion Transformers Upcited by 28
  35. 2026InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editingcited by 28
  36. 2024SnapGen: Taming High-Resolution Text-to-Image Models for Mobile Devices with Efficient Architectures and Trainingcited by 27
  37. 2024SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Devicecited by 27
  38. 2025WorldExplorer: Towards Generating Fully Navigable 3D Scenescited by 26

50 most cited of 340 All 340 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis
  2. 2026Efficient One-Step Diffusion Restoration Model with Compact Token Compression and Linear Attention
  3. 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer
  4. 2024High-Resolution Image Synthesis via Next-Token Prediction
  5. 2025UltraImageGen: Efficient Ultra-High-Resolution Image Generation with Hierarchical Local Attention
  6. 2025Text-Aware Real-World Image Super-Resolution via Diffusion Model with Joint Segmentation Decoders
  7. 2023PanGu-Draw: Advancing Resource-Efficient Text-to-Image Synthesis with Time-Decoupled Training and Reusable Coop-Diffusion
  8. 2026TOLA: Text-aware One-Step Latent Adaptation for Diffusion-based Text Image Super-Resolution