mlsanityData to 7 Oct 2026

EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling

arXiv 2502.09509 ↗First posted 13 Feb 2025cs.LGcited by 85

Not on the map. Why?

Cited by 54 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Latent generative models have emerged as a leading approach for high-quality image synthesis. These models rely on an autoencoder to compress images into a latent space, followed by a generative model to learn the latent distribution. We identify that existing autoencoders lack equivariance to semantic-preserving transformations like scaling and rotation, resulting in complex latent spaces that hinder generative performance. To address this, we propose EQ-VAE, a simple regularization approach that enforces equivariance in the latent space, reducing its complexity without degrading reconstruction quality. By finetuning pre-trained autoencoders with EQ-VAE, we enhance the performance of several state-of-the-art generative models, including DiT, SiT, REPA and MaskGIT, achieving a 7 speedup on DiT-XL/2 with only five epochs of SD-VAE fine-tuning. EQ-VAE is compatible with both continuous and discrete autoencoders, thus offering a versatile enhancement for a wide range of latent generative models. Project page and code: https://eq-vae.github.io/.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Builds on 58 of its 74 references that are in the map. First posted on arXiv 13 Feb 2025.

Builds on

58 of 74 references.

References · 58

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2022Scalable Diffusion Models with Transformerscited by 5,931
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  7. 2017Neural Discrete Representation Learningcited by 4,650
  8. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  11. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  12. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
38 more
  1. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  2. 2016Improved Techniques for Training GANscited by 2,938
  3. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  4. 2016Group Equivariant Convolutional Networkscited by 1,154
  5. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  6. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  7. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  8. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  9. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  10. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  11. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  12. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  13. 2019General E(2)-Equivariant Steerable CNNscited by 441
  14. 2017Wasserstein Auto-Encoderscited by 371
  15. 2015Adversarial Autoencoderscited by 359
  16. 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 307
  17. 2021Generating Images with Sparse Representationscited by 262
  18. 2017VAE with a VampPriorcited by 247
  19. 2018Estimating the intrinsic dimension of datasets by a minimal neighborhood informationcited by 246
  20. 2024Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Modelscited by 240
  21. 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 222
  22. 2016Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoderscited by 219
  23. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 209
  24. 2023Fast Training of Diffusion Models with Masked Transformerscited by 172
  25. 2021Frame Averaging for Invariant and Equivariant Network Designcited by 166
  26. 2017Equivariance Through Parameter-Sharingcited by 139
  27. 2019Scale-Equivariant Steerable Networkscited by 110
  28. 2023The geometry of hidden representations of large transformer modelscited by 108
  29. 2016Rotation equivariant vector field networkscited by 98
  30. 2023GIVT: Generative Infinite-Vocabulary Transformerscited by 80
  31. 2024Scaling the Codebook Size of VQGAN to 100,000 with a Utilization Rate of 99%cited by 68
  32. 2024FasterDiT: Towards Faster Diffusion Transformers Training without Architecture Modificationcited by 64
  33. 2018CubeNet: Equivariance to 3D Rotation and Translationcited by 59
  34. 2021Consistency Regularization for Variational Auto-Encoderscited by 50
  35. 2024Does equivariance matter at scale?cited by 49
  36. 2024SD-DiT: Unleashing the Power of Self-supervised Discrimination in Diffusion Transformercited by 47
  37. 2023Designing a Better Asymmetric VQGAN for StableDiffusioncited by 45
  38. 2018N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentialscited by 42

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 43; 2026: 42.

85 papers.

Sort:

By area:

  1. 2025Diffusion Transformers with Representation Autoencoderscited by 275
  2. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 212
  3. 2025Latent Diffusion Model without Variational Autoencodercited by 83
  4. 2025Improving the Diffusability of Autoencoderscited by 74
  5. 2025DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Spacecited by 44
  6. 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesiscited by 42
  7. 2025AlignTok: Aligning Visual Foundation Encoders to Tokenizers for Diffusion Modelscited by 35
  8. 2026Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generationcited by 33
  9. 2025USP: Unified Self-Supervised Pretraining for Image Generation and Understandingcited by 24
  10. 2025Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modelingcited by 24
  11. 2025Both Semantics and Reconstruction Matter: Making Representation Encoders Ready for Text-to-Image Generation and Editingcited by 22
  12. 2025Orbis: Overcoming Challenges of Long-Horizon Prediction in Driving World Modelscited by 17
38 more
  1. 2025Selective Underfitting in Diffusion Modelscited by 17
  2. 2025VFM-VAE: Vision Foundation Models Can Be Good Tokenizers for Latent Diffusion Modelscited by 16
  3. 2025SVG-T2I: Scaling Up Text-to-Image Latent Diffusion Model Without Variational Autoencodercited by 12
  4. 2026Demystifying Data-Driven Probabilistic Medium-Range Weather Forecastingcited by 12
  5. 2026Making Reconstruction FID Predictive of Diffusion Generation FIDcited by 12
  6. 2025Delving into Latent Spectral Biasing of Video VAEs for Superior Diffusabilitycited by 10
  7. 2026Improving Reconstruction of Representation Autoencodercited by 8
  8. 2025Decoupling High and Low Frequencies for Faithful Image Generation with Fine Detailscited by 7
  9. 2025DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Spacecited by 7
  10. 2025Toward Diffusible High-Dimensional Latent Spaces: A Frequency Perspectivecited by 7
  11. 2026What Matters for Diffusion-Friendly Latent Manifold? Prior-Aligned Autoencoders for Latent Diffusioncited by 7
  12. 2025Enhancing Variational Autoencoders with Smooth Robust Latent Encodingcited by 6
  13. 2025Latent Wavelet Diffusion For Ultra-High-Resolution Image Synthesiscited by 6
  14. 2025Improving Progressive Generation with Decomposable Flow Matchingcited by 6
  15. 2025SuperGen: An Efficient Ultra-high-resolution Video Generation System with Sketching and Tilingcited by 6
  16. 2025SimFlow: Simplified and End-to-End Training of Latent Normalizing Flowscited by 6
  17. 2026Spectrum Matching: a Unified Perspective for Superior Diffusability in Latent Diffusioncited by 6
  18. 2025Distribution Matching Variational AutoEncodercited by 5
  19. 2025VFMF: World Modeling by Forecasting Vision Foundation Model Featurescited by 5
  20. 2025REGLUE Your Latents with Global and Local Semantics for Entangled Diffusioncited by 5
  21. 2026FSVideo: Fast Speed Video Diffusion Model in a Highly-Compressed Latent Spacecited by 5
  22. 2026Laplacian Multi-scale Flow Matching for Generative Modelingcited by 5
  23. 2026Diffusing in the Right Space: A Systematic Study of Latent Diffusabilitycited by 5
  24. 2025On Equivariance and Fast Sampling in Video Diffusion Models Trained with Warped Noisecited by 4
  25. 2025Re-Bottleneck: Latent Re-Structuring for Neural Audio Autoencoderscited by 4
  26. 2025VoiceBridge: General Speech Restoration with One-step Latent Bridge Modelscited by 4
  27. 2025Latent Harmony: Synergistic Unified UHD Image Restoration via Latent Space Regularization and Controllable Refinementcited by 3
  28. 2025Cloning Deterministic Worlds: The Critical Role of Latent Geometry in Long-Horizon World Modelscited by 3
  29. 2025FlowEO: Generative Unsupervised Domain Adaptation for Earth Observationcited by 3
  30. 2026Steering Visual Generation in Unified Multimodal Models with Understanding Supervisioncited by 3
  31. 2025gen2seg: Generative Models Enable Generalizable Instance Segmentationcited by 2
  32. 2025VIVAT: Virtuous Improving VAE Training through Artifact Mitigationcited by 2
  33. 2025Splatent: Splatting Diffusion Latents for Novel View Synthesiscited by 2
  34. 2026Approximate Equivariance via Projection-based Regularisationcited by 2
  35. 2026MPDiT: Multi-Patch Global-to-Local Transformer Architecture For Efficient Flow Matching and Diffusion Modelcited by 2
  36. 2026Aligning Latent Geometry for Spherical Flow Matching in Image Generationcited by 2
  37. 2026RiT: Vanilla Diffusion Transformers Suffice in Representation Spacecited by 2
  38. 2025Optimal Stopping in Latent Diffusion Modelscited by 1

50 most cited of 85 All 85 (JSON)