# EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling

arXiv: https://arxiv.org/abs/2502.09509 · HTML: https://mlsanity.com/paper/2502.09509/

## Key facts

- Cited by 89 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 74 references that are in the map.
- First posted on arXiv 13 Feb 2025.

## Builds on (most cited first, 30 of 58)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction](https://mlsanity.com/paper/2404.02905/) (2024)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [Autoregressive Image Generation without Vector Quantization](https://mlsanity.com/paper/2406.11838/) (2024)
- [Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation](https://mlsanity.com/paper/2310.05737/) (2023)
- [General E(2)-Equivariant Steerable CNNs](https://mlsanity.com/paper/1911.08251/) (2019)
- [Wasserstein Auto-Encoders](https://mlsanity.com/paper/1711.01558/) (2017)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Generating Images with Sparse Representations](https://mlsanity.com/paper/2103.03841/) (2021)
- [Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models](https://mlsanity.com/paper/2404.07724/) (2024)

## Built on by (most cited first, 30 of 89)

- [Diffusion Transformers with Representation Autoencoders](https://mlsanity.com/paper/2510.11690/) (2025)
- [REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformers](https://mlsanity.com/paper/2504.10483/) (2025)
- [Latent Diffusion Model without Variational Autoencoder](https://mlsanity.com/paper/2510.15301/) (2025)
- [Improving the Diffusability of Autoencoders](https://mlsanity.com/paper/2502.14831/) (2025)
- [DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Space](https://mlsanity.com/paper/2508.00413/) (2025)
- [Boosting Generative Image Modeling via Joint Image-Feature Synthesis](https://mlsanity.com/paper/2504.16064/) (2025)
- [AlignTok: Aligning Visual Foundation Encoders to Tokenizers for Diffusion Models](https://mlsanity.com/paper/2509.25162/) (2025)
- [Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generation](https://mlsanity.com/paper/2602.11401/) (2026)
- [Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modeling](https://mlsanity.com/paper/2506.21170/) (2025)
- [USP: Unified Self-Supervised Pretraining for Image Generation and Understanding](https://mlsanity.com/paper/2503.06132/) (2025)
- [Both Semantics and Reconstruction Matter: Making Representation Encoders Ready for Text-to-Image Generation and Editing](https://mlsanity.com/paper/2512.17909/) (2025)
- [Selective Underfitting in Diffusion Models](https://mlsanity.com/paper/2510.01378/) (2025)
- [Orbis: Overcoming Challenges of Long-Horizon Prediction in Driving World Models](https://mlsanity.com/paper/2507.13162/) (2025)
- [VFM-VAE: Vision Foundation Models Can Be Good Tokenizers for Latent Diffusion Models](https://mlsanity.com/paper/2510.18457/) (2025)
- [Demystifying Data-Driven Probabilistic Medium-Range Weather Forecasting](https://mlsanity.com/paper/2601.18111/) (2026)
- [Making Reconstruction FID Predictive of Diffusion Generation FID](https://mlsanity.com/paper/2603.05630/) (2026)
- [SVG-T2I: Scaling Up Text-to-Image Latent Diffusion Model Without Variational Autoencoder](https://mlsanity.com/paper/2512.11749/) (2025)
- [Delving into Latent Spectral Biasing of Video VAEs for Superior Diffusability](https://mlsanity.com/paper/2512.05394/) (2025)
- [Toward Diffusible High-Dimensional Latent Spaces: A Frequency Perspective](https://mlsanity.com/paper/2511.22249/) (2025)
- [Improving Reconstruction of Representation Autoencoder](https://mlsanity.com/paper/2602.08620/) (2026)
- [What Matters for Diffusion-Friendly Latent Manifold? Prior-Aligned Autoencoders for Latent Diffusion](https://mlsanity.com/paper/2605.07915/) (2026)
- [DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space](https://mlsanity.com/paper/2509.25180/) (2025)
- [Latent Wavelet Diffusion For Ultra-High-Resolution Image Synthesis](https://mlsanity.com/paper/2506.00433/) (2025)
- [Decoupling High and Low Frequencies for Faithful Image Generation with Fine Details](https://mlsanity.com/paper/2509.05441/) (2025)
- [SimFlow: Simplified and End-to-End Training of Latent Normalizing Flows](https://mlsanity.com/paper/2512.04084/) (2025)
- [Spectrum Matching: a Unified Perspective for Superior Diffusability in Latent Diffusion](https://mlsanity.com/paper/2603.14645/) (2026)
- [Enhancing Variational Autoencoders with Smooth Robust Latent Encoding](https://mlsanity.com/paper/2504.17219/) (2025)
- [Improving Progressive Generation with Decomposable Flow Matching](https://mlsanity.com/paper/2506.19839/) (2025)
- [SuperGen: An Efficient Ultra-high-resolution Video Generation System with Sketching and Tiling](https://mlsanity.com/paper/2508.17756/) (2025)
- [Distribution Matching Variational AutoEncoder](https://mlsanity.com/paper/2512.07778/) (2025)

## Abstract (arXiv)

Latent generative models have emerged as a leading approach for high-quality image synthesis. These models rely on an autoencoder to compress images into a latent space, followed by a generative model to learn the latent distribution. We identify that existing autoencoders lack equivariance to semantic-preserving transformations like scaling and rotation, resulting in complex latent spaces that hinder generative performance. To address this, we propose EQ-VAE, a simple regularization approach that enforces equivariance in the latent space, reducing its complexity without degrading reconstruction quality. By finetuning pre-trained autoencoders with EQ-VAE, we enhance the performance of several state-of-the-art generative models, including DiT, SiT, REPA and MaskGIT, achieving a 7 speedup on DiT-XL/2 with only five epochs of SD-VAE fine-tuning. EQ-VAE is compatible with both continuous and discrete autoencoders, thus offering a versatile enhancement for a wide range of latent generative models. Project page and code: https://eq-vae.github.io/.