mlsanityData to 7 Oct 2026

Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models

arXiv 2410.10733 ↗First posted 14 Oct 2024cs.CV cs.AIcited by 209

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We present Deep Compression Autoencoder (DC-AE), a new family of autoencoder models for accelerating high-resolution diffusion models. Existing autoencoder models have demonstrated impressive results at a moderate spatial compression ratio (e.g., 8x), but fail to maintain satisfactory reconstruction accuracy for high spatial compression ratios (e.g., 64x). We address this challenge by introducing two key techniques: (1) Residual Autoencoding, where we design our models to learn residuals based on the space-to-channel transformed features to alleviate the optimization difficulty of high spatial-compression autoencoders; (2) Decoupled High-Resolution Adaptation, an efficient decoupled three-phases training strategy for mitigating the generalization penalty of high spatial-compression autoencoders. With these designs, we improve the autoencoder's spatial compression ratio up to 128 while maintaining the reconstruction quality. Applying our DC-AE to latent diffusion models, we achieve significant speedup without accuracy drop. For example, on ImageNet 512x512, our DC-AE provides 19.1x inference speedup and 17.9x training speedup on H100 GPU for UViT-H while achieving a better FID, compared with the widely used SD-VAE-f8 autoencoder. Our code is available at https://github.com/mit-han-lab/efficientvit.

Key facts. Cited by 209 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 49 of its 58 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

49 of 58 references.

Closest ancestors · 30

Same direction, newest first

  1. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation4 Jun
  2. 2024Improved Distribution Matching Distillation for Fast Image Synthesis23 May
  3. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation7 Mar
  4. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
  5. 2024DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models29 Feb
  6. 2024Accelerating Parallel Sampling of Diffusion Models15 Feb
  7. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers16 Jan
  8. 2023Analyzing and Improving the Training Dynamics of Diffusion Models5 Dec
  9. 2023DeepCache: Accelerating Diffusion Models for Free1 Dec
  10. 2023One-step Diffusion with Distribution Matching Distillation30 Nov
  11. 2023DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics20 Oct
  12. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference6 Oct
18 more
  1. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis30 Sep
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  3. 2023Parallel Sampling of Diffusion Models25 May
  4. 2023Structural Pruning for Diffusion Models18 May
  5. 2023PTQD: Accurate Post-Training Quantization for Diffusion Models18 May
  6. 2023Consistency Models2 Mar
  7. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models9 Feb
  8. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  9. 2022Scalable Diffusion Models with Transformers19 Dec
  10. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  11. 2022On Distillation of Guided Diffusion Models6 Oct
  12. 2022All are Worth Words: A ViT Backbone for Diffusion Models25 Sep
  13. 2022gDDIM: Generalized denoising diffusion implicit models11 Jun
  14. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  15. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  16. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  17. 2020Denoising Diffusion Implicit Models6 Oct
  18. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 19

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2023Segment Anythingcited by 8,158
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  6. 2016Improved Techniques for Training GANscited by 2,938
  7. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  8. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  9. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  10. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  11. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  12. 2023Rethinking FID: Towards a Better Evaluation Metric for Image Generationcited by 303
7 more
  1. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 278
  2. 2024Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generationcited by 249
  3. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
  4. 2023Designing a Better Asymmetric VQGAN for StableDiffusioncited by 45
  5. 2022Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Modelscited by 44
  6. 2024LinFusion: 1 GPU, 1 Minute, 16K Imagecited by 42
  7. 2024Condition-Aware Neural Network for Controlled Image Generationcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 107; 2026: 96.

209 papers.

Sort:

By area:

  1. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  2. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 376
  3. 2025Diffusion Transformers with Representation Autoencoderscited by 275
  4. 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 247
  5. 2025Motus: A Unified Latent Action World Modelcited by 231
  6. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsitycited by 204
  7. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 178
  8. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  9. 2025Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Modelcited by 163
  10. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 161
  11. 2025Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200kcited by 157
  12. 2025Native and Compact Structured Latents for 3D Generationcited by 157
38 more
  1. 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 119
  2. 2025Epona: Autoregressive Diffusion World Model for Autonomous Drivingcited by 115
  3. 2025Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutationcited by 100
  4. 2024Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoderscited by 93
  5. 2025Masked Autoencoders Are Effective Tokenizers for Diffusion Modelscited by 93
  6. 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformercited by 91
  7. 2025EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modelingcited by 85
  8. 2025SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillationcited by 85
  9. 2024FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matchingcited by 77
  10. 2025PixNerd: Pixel Neural Field Diffusioncited by 75
  11. 2025PixelDiT: Pixel Diffusion Transformers for Image Generationcited by 75
  12. 2025Improving the Diffusability of Autoencoderscited by 74
  13. 2025Diffusion-4K: Ultra-High-Resolution Image Synthesis with Latent Diffusion Modelscited by 61
  14. 2025UniVerse-1: Unified Audio-Video Generation via Stitching of Expertscited by 58
  15. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 55
  16. 2025DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generationcited by 55
  17. 2025ACE-Step: A Step Towards Music Generation Foundation Modelcited by 54
  18. 2025ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinementcited by 53
  19. 2025BLIP3o-NEXT: Next Frontier of Native Image Generationcited by 50
  20. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 50
  21. 2026Scaling Text-to-Image Diffusion Transformers with Representation Autoencoderscited by 49
  22. 2025OpenUni: A Simple Baseline for Unified Multimodal Understanding and Generationcited by 45
  23. 2025DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Spacecited by 44
  24. 2025UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editingcited by 43
  25. 2025DiP: Taming Diffusion Models in Pixel Spacecited by 42
  26. 2025Flow to the Mode: Mode-Seeking Diffusion Autoencoders for State-of-the-Art Image Tokenizationcited by 35
  27. 2024Switti: Designing Scale-Wise Transformers for Text-to-Image Synthesiscited by 33
  28. 2026Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generationcited by 33
  29. 2025Transition Models: Rethinking the Generative Learning Objectivecited by 32
  30. 2025Bridging Continuous and Discrete Tokens for Autoregressive Visual Generationcited by 29
  31. 2026Qwen-Image-2.0 Technical Reportcited by 28
  32. 2025Ming-UniVision: Joint Image Understanding and Generation with a Unified Continuous Tokenizercited by 27
  33. 2025StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generationcited by 27
  34. 2025An Analytical Theory of Spectral Bias in the Learning Dynamics of Diffusion Modelscited by 26
  35. 2025Unleashing Vecset Diffusion Model for Fast Shape Generationcited by 25
  36. 2025A Survey of Interactive Generative Videocited by 23
  37. 2025Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminatorcited by 22
  38. 2025DraftAttention: Fast Video Diffusion via Low-Resolution Attention Guidancecited by 21

50 most cited of 209 All 209 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Models
  2. 2026Addressing Detail Bottlenecks in Latent Diffusion for RGB-to-SWIR Image Translation
  3. 2024LiteVAE: Lightweight and Efficient Variational Autoencoders for Latent Diffusion Models
  4. 2024PaGoDA: Progressive Growing of a One-Step Generator from a Low-Resolution Diffusion Teacher
  5. 2024Sample what you cant compress
  6. 2024Boosting Latent Diffusion with Perceptual Objectives
  7. 2023Variational Diffusion Auto-encoder: Latent Space Extraction from Pre-trained Diffusion Models
  8. 2023Denoising Diffusion Autoencoders are Unified Self-supervised Learners