# DA-VAE: Plug-in Latent Compression for Diffusion via Detail Alignment arXiv: https://arxiv.org/abs/2603.22125 · HTML: https://mlsanity.com/paper/2603.22125/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. - Builds on 50 of its 57 references that are in the map. - First posted on arXiv 23 Mar 2026. ## Placed in - Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/) ## Builds on (most cited first, 30 of 50) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018) - [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022) - [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023) - [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024) - [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023) - [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023) - [Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://mlsanity.com/paper/2310.04378/) (2023) - [On Distillation of Guided Diffusion Models](https://mlsanity.com/paper/2210.03142/) (2022) - [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024) - [Improved Distribution Matching Distillation for Fast Image Synthesis](https://mlsanity.com/paper/2405.14867/) (2024) - [Mean Flows for One-step Generative Modeling](https://mlsanity.com/paper/2505.13447/) (2025) - [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022) - [All are Worth Words: A ViT Backbone for Diffusion Models](https://mlsanity.com/paper/2209.12152/) (2022) - [UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models](https://mlsanity.com/paper/2302.04867/) (2023) - [Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models](https://mlsanity.com/paper/2501.01423/) (2025) - [One Step Diffusion via Shortcut Models](https://mlsanity.com/paper/2410.12557/) (2024) - [DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models](https://mlsanity.com/paper/2210.14896/) (2022) - [DeepCache: Accelerating Diffusion Models for Free](https://mlsanity.com/paper/2312.00858/) (2023) - [SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers](https://mlsanity.com/paper/2410.10629/) (2024) - [An Image is Worth 32 Tokens for Reconstruction and Generation](https://mlsanity.com/paper/2406.07550/) (2024) - [Diffusion Transformers with Representation Autoencoders](https://mlsanity.com/paper/2510.11690/) (2025) - [Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystack](https://mlsanity.com/paper/2309.15807/) (2023) - [Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generation](https://mlsanity.com/paper/2402.17245/) (2024) - [Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models](https://mlsanity.com/paper/2410.10733/) (2024) - [Structural Pruning for Diffusion Models](https://mlsanity.com/paper/2305.10924/) (2023) - [SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models](https://mlsanity.com/paper/2411.05007/) (2024) ## Built on by (most cited first, 6 of 6) - [L2P: Unlocking Latent Potential for Pixel Generation](https://mlsanity.com/paper/2605.12013/) (2026) - [Twins: Learn to Predict Unified Representations with Focal Loss](https://mlsanity.com/paper/2607.22531/) (2026) - [Parallax Portrait Matting](https://mlsanity.com/paper/2607.11205/) (2026) - [LatentQuant: Preserving the Policy-Facing Latent Contract under NVFP4 VAE Quantization](https://mlsanity.com/paper/2610.03959/) (2026) - [GRACE: Generation-aware latent compression for efficient video generation](https://mlsanity.com/paper/2610.10524/) (2026) - [APEX: Asynchronous Prefix Denoising for Extreme Compression](https://mlsanity.com/paper/2610.12843/) (2026) ## Abstract (arXiv) Reducing token count is crucial for efficient training and inference of latent diffusion models, especially at high resolution. A common strategy is to build high-compression image tokenizers with more channels per token. However, when trained only for reconstruction, high-dimensional latent spaces often lose meaningful structure, making diffusion training harder. Existing methods address this with extra objectives such as semantic alignment or selective dropout, but usually require costly diffusion retraining. Pretrained diffusion models, however, already exhibit a structured, lower-dimensional latent space; thus, a simpler idea is to expand the latent dimensionality while preserving this structure. We therefore propose Detail-Aligned VAE, which increases the compression ratio of a pretrained VAE with only lightweight adaptation of the pretrained diffusion backbone. DA-VAE uses an explicit latent layout: the first C channels come directly from the pretrained VAE at a base resolution, while an additional D channels encode higher-resolution details. A simple detail-alignment mechanism encourages the expanded latent space to retain the structure of the original one. With a warm-start fine-tuning strategy, our method enables 1024 × 1024 image generation with Stable Diffusion 3.5 using only 32 × 32 tokens, 4× fewer than the original model, within 5 H100-days. It further unlocks 2048 × 2048 generation with SD3.5, achieving a 6× speedup while preserving image quality. We also validate the method and its design choices quantitatively on ImageNet.