# Denoising Vision Transformer Autoencoder with Spectral Self-Regularization

arXiv: https://arxiv.org/abs/2511.12633 · HTML: https://mlsanity.com/paper/2511.12633/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- First posted on arXiv 16 Nov 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Built on by (most cited first, 7 of 7)

- [What Matters for Diffusion-Friendly Latent Manifold? Prior-Aligned Autoencoders for Latent Diffusion](https://mlsanity.com/paper/2605.07915/) (2026)
- [PhysOmni: Physics-Grounded Multi-Object Scene Generation from a Single Image with Real-Time Interaction](https://mlsanity.com/paper/2605.20290/) (2026)
- [PuYun-LDM: A Latent Diffusion Model for High-Resolution Ensemble Weather Forecasts](https://mlsanity.com/paper/2602.11807/) (2026)
- [UniCSG: Unified High-Fidelity Content-Constrained Style-Driven Generation via Staged Semantic and Frequency Disentanglement](https://mlsanity.com/paper/2604.17850/) (2026)
- [Spectral Vision Transformer for Efficient Tokenization with Limited Data](https://mlsanity.com/paper/2605.12026/) (2026)
- [APEX: Asynchronous Prefix Denoising for Extreme Compression](https://mlsanity.com/paper/2610.12843/) (2026)
- [Spectral Considerations on Unsupervised Time-domain EEG Representation Learning with Deep Networks](https://mlsanity.com/paper/2610.16264/) (2026)

## Abstract (arXiv)

Variational autoencoders (VAEs) typically encode images into a compact latent space, reducing computational cost but introducing an optimization dilemma: a higher-dimensional latent space improves reconstruction fidelity but often hampers generative performance. Recent methods attempt to address this dilemma by regularizing high-dimensional latent spaces using external vision foundation models (VFMs). However, it remains unclear how high-dimensional VAE latents affect the optimization of generative models. To our knowledge, our analysis is the first to reveal that redundant high-frequency components in high-dimensional latent spaces hinder the training convergence of diffusion models and, consequently, degrade generation quality. To alleviate this problem, we propose a spectral self-regularization strategy to suppress redundant high-frequency noise while simultaneously preserving reconstruction quality. The resulting Denoising-VAE, a ViT-based autoencoder that does not rely on VFMs, produces cleaner, lower-noise latents, leading to improved generative quality and faster optimization convergence. We further introduce a spectral alignment strategy to facilitate the optimization of Denoising-VAE-based generative models. Our complete method enables diffusion models to converge approximately 2× faster than with SD-VAE, while achieving state-of-the-art reconstruction quality (rFID = 0.28, PSNR = 27.26) and competitive generation performance (gFID = 1.82) on the ImageNet 256×256 benchmark.