# SnapGen: Taming High-Resolution Text-to-Image Models for Mobile Devices with Efficient Architectures and Training

arXiv: https://arxiv.org/abs/2412.09619 · HTML: https://mlsanity.com/paper/2412.09619/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 64 of its 79 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)
- Knowledge distillation / [Multimodal distillation](https://mlsanity.com/distill/multimodal-distillation/)

## Builds on (most cited first, 30 of 64)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [Adversarial Diffusion Distillation](https://mlsanity.com/paper/2311.17042/) (2023)
- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [Fast Transformer Decoding: One Write-Head is All You Need](https://mlsanity.com/paper/1911.02150/) (2019)
- [Improved Distribution Matching Distillation for Fast Image Synthesis](https://mlsanity.com/paper/2405.14867/) (2024)

## Built on by (most cited first, 27 of 27)

- [SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer](https://mlsanity.com/paper/2501.18427/) (2025)
- [SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillation](https://mlsanity.com/paper/2503.09641/) (2025)
- [H3AE: High Compression, High Speed, and High Quality AutoEncoder for Video Diffusion Models](https://mlsanity.com/paper/2504.10567/) (2025)
- [Taming Diffusion Transformer for Efficient Mobile Video Generation in Seconds](https://mlsanity.com/paper/2507.13343/) (2025)
- [HierarchicalPrune: Position-Aware Compression for Large-Scale Diffusion Models](https://mlsanity.com/paper/2508.04663/) (2025)
- [PocketSR: The Super-Resolution Expert in Your Pocket Mobiles](https://mlsanity.com/paper/2510.03012/) (2025)
- [TinySR: Pruning Diffusion for Real-World Image Super-Resolution](https://mlsanity.com/paper/2508.17434/) (2025)
- [MobileI2V: Fast and High-Resolution Image-to-Video on Mobile Devices](https://mlsanity.com/paper/2511.21475/) (2025)
- [TP-Blend: Textual-Prompt Attention Pairing for Precise Object-Style Blending in Diffusion Models](https://mlsanity.com/paper/2601.08011/) (2026)
- [DDiT: Dynamic Patch Scheduling for Efficient Diffusion Transformers](https://mlsanity.com/paper/2602.16968/) (2026)
- [DreamLite: A Lightweight On-Device Unified Model for Image Generation and Editing](https://mlsanity.com/paper/2603.28713/) (2026)
- [GuideSR: Rethinking Guidance for One-Step High-Fidelity Diffusion-Based Super-Resolution](https://mlsanity.com/paper/2505.00687/) (2025)
- [Consistent and Controllable Image Animation with Motion Linear Diffusion Transformers](https://mlsanity.com/paper/2508.07246/) (2025)
- [SnapGen++: Unleashing Diffusion Transformers for Efficient High-Fidelity Image Generation on Edge Devices](https://mlsanity.com/paper/2601.08303/) (2026)
- [TurboVSR: Fantastic Video Upscalers and Where to Find Them](https://mlsanity.com/paper/2506.23618/) (2025)
- [NanoSD: Edge Efficient Foundation Model for Real Time Image Restoration](https://mlsanity.com/paper/2601.09823/) (2026)
- [Eliminating VAE for Fast and High-Resolution Generative Detail Restoration](https://mlsanity.com/paper/2602.10630/) (2026)
- [WorldAgents: Can Foundation Image Models be Agents for 3D World Models?](https://mlsanity.com/paper/2603.19708/) (2026)
- [Cross-Space Distillation: Teaching One-Step Students with Modern Diffusion Teachers](https://mlsanity.com/paper/2606.32020/) (2026)
- [SlimDiff: Training-Free, Activation-Guided Hands-free Slimming of Diffusion Models](https://mlsanity.com/paper/2509.21498/) (2025)
- [ArtifactLens: Hundreds of Labels Are Enough for Artifact Detection with VLMs](https://mlsanity.com/paper/2602.09475/) (2026)
- [Mobile-O: Unified Multimodal Understanding and Generation on Mobile Device](https://mlsanity.com/paper/2602.20161/) (2026)
- [BlazeEdit: Generalist Image Editing on Mobile Devices with Image-to-Image Diffusion Models](https://mlsanity.com/paper/2605.28067/) (2026)
- [REFINE-CONTROL: A Semi-supervised Distillation Method For Conditional Image Generation](https://mlsanity.com/paper/2509.22139/) (2025)
- [Reviving ConvNeXt for Efficient Convolutional Diffusion Models](https://mlsanity.com/paper/2603.09408/) (2026)
- [ElasticDiT: Efficient Diffusion Transformers via Elastic Architecture and Sparse Attention for High-Resolution Image Generation on Mobile Devices](https://mlsanity.com/paper/2605.15684/) (2026)
- [JuZhou 1.0 Technical Report: The First Edge-Native Text-to-Image Foundation Model Trained Entirely on China-Developed AI Accelerators](https://mlsanity.com/paper/2606.28421/) (2026)

## Abstract (arXiv)

Existing text-to-image (T2I) diffusion models face several limitations, including large model sizes, slow runtime, and low-quality generation on mobile devices. This paper aims to address all of these challenges by developing an extremely small and fast T2I model that generates high-resolution and high-quality images on mobile platforms. We propose several techniques to achieve this goal. First, we systematically examine the design choices of the network architecture to reduce model parameters and latency, while ensuring high-quality generation. Second, to further improve generation quality, we employ cross-architecture knowledge distillation from a much larger model, using a multi-level approach to guide the training of our model from scratch. Third, we enable a few-step generation by integrating adversarial guidance with knowledge distillation. For the first time, our model SnapGen, demonstrates the generation of 1024x1024 px images on a mobile device around 1.4 seconds. On ImageNet-1K, our model, with only 372M parameters, achieves an FID of 2.06 for 256x256 px generation. On T2I benchmarks (i.e., GenEval and DPG-Bench), our model with merely 379M parameters, surpasses large-scale models with billions of parameters at a significantly smaller size (e.g., 7x smaller than SDXL, 14x smaller than IF-XL).