# Towards Scalable Pre-training of Visual Tokenizers for Generation

arXiv: https://arxiv.org/abs/2512.13687 · HTML: https://mlsanity.com/paper/2512.13687/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2025.

## Built on by (most cited first, 28 of 28)

- [Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders](https://mlsanity.com/paper/2601.16208/) (2026)
- [Making Reconstruction FID Predictive of Diffusion Generation FID](https://mlsanity.com/paper/2603.05630/) (2026)
- [What Matters for Diffusion-Friendly Latent Manifold? Prior-Aligned Autoencoders for Latent Diffusion](https://mlsanity.com/paper/2605.07915/) (2026)
- [Lens: Rethinking Training Efficiency for Foundational Text-to-Image Models](https://mlsanity.com/paper/2605.21573/) (2026)
- [Diffusing in the Right Space: A Systematic Study of Latent Diffusability](https://mlsanity.com/paper/2606.03578/) (2026)
- [An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Models](https://mlsanity.com/paper/2608.16887/) (2026)
- [GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation](https://mlsanity.com/paper/2609.05588/) (2026)
- [AniGen: Unified S3 Fields for Animatable 3D Asset Generation](https://mlsanity.com/paper/2604.08746/) (2026)
- [MUSE: Resolving Manifold Misalignment in Visual Tokenization via Topological Orthogonality](https://mlsanity.com/paper/2605.05646/) (2026)
- [WavCube: Unifying Speech Representation for Understanding and Generation via Semantic-Acoustic Joint Modeling](https://mlsanity.com/paper/2605.06407/) (2026)
- [Towards Predictive, Aligned, and Scalable Robot Learning](https://mlsanity.com/paper/2607.11270/) (2026)
- [Geometric Autoencoder for Diffusion Models](https://mlsanity.com/paper/2603.10365/) (2026)
- [HYDRA: Representation Harmonized Tokenization for Multimodal Generation and Understanding](https://mlsanity.com/paper/2603.15228/) (2026)
- [Qwen-Image-VAE-2.0 Technical Report](https://mlsanity.com/paper/2605.13565/) (2026)
- [HYDRA-X: Native Unified Multimodal Models with Holistic Visual Tokenizers](https://mlsanity.com/paper/2606.13289/) (2026)
- [Semantic-Aware Prefix Learning for Token-Efficient Image Generation](https://mlsanity.com/paper/2603.25249/) (2026)
- [TC-AE: Unlocking Token Capacity for Deep Compression Autoencoders](https://mlsanity.com/paper/2604.07340/) (2026)
- [MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image Generation](https://mlsanity.com/paper/2606.26016/) (2026)
- [UniSpace: Unified Visual Representation and Scalable Multimodal Modeling](https://mlsanity.com/paper/2608.08676/) (2026)
- [A Tilted Seesaw: Revisiting Autoencoder Trade-off for Controllable Diffusion](https://mlsanity.com/paper/2601.21633/) (2026)
- [UniCSG: Unified High-Fidelity Content-Constrained Style-Driven Generation via Staged Semantic and Frequency Disentanglement](https://mlsanity.com/paper/2604.17850/) (2026)
- [Video Generation with Predictive Latents](https://mlsanity.com/paper/2605.02134/) (2026)
- [The Impact of VAE Design on Latent Pose Representations for Diffusion-based Sign Language Production](https://mlsanity.com/paper/2606.22959/) (2026)
- [dRAE: Representation Autoencoder with Hyper-Spherical Codes](https://mlsanity.com/paper/2607.22148/) (2026)
- [OmniVAE: An Audio-Video VAE with Cross-Modal Alignment for Joint Generation](https://mlsanity.com/paper/2607.23855/) (2026)
- [Multimodal Flow: Unified Flow Modeling of Language and Vision in Embedding Spaces](https://mlsanity.com/paper/2609.40362/) (2026)
- [VGGA: Geometry-Grounded Autoencoder for Video Generation](https://mlsanity.com/paper/2610.16204/) (2026)
- [Generative Representation Learning for Scalable Latent Tokenization](https://mlsanity.com/paper/2610.20533/) (2026)

## Abstract (arXiv)

The quality of the latent space in visual tokenizers (e.g., VAEs) is crucial for modern generative models. However, the standard reconstruction-based training paradigm produces a latent space that is biased towards low-level information, leading to a foundation flaw: better pixel-level accuracy does not lead to higher-quality generation. This implies that pouring extensive compute into visual tokenizer pre-training translates poorly to improved performance in generation. We identify this as the ``pre-training scaling problem`` and suggest a necessary shift: to be effective for generation, a latent space must concisely represent high-level semantics. We present VTP, a unified visual tokenizer pre-training framework, pioneering the joint optimization of image-text contrastive, self-supervised, and reconstruction losses. Our large-scale study reveals two principal findings: (1) understanding is a key driver of generation, and (2) much better scaling properties, where generative performance scales effectively with compute, parameters, and data allocated to the pretraining of the visual tokenizer. After large-scale pre-training, our tokenizer delivers a competitive profile (78.2 zero-shot accuracy and 0.36 rFID on ImageNet) and 4.1 times faster convergence on generation compared to advanced distillation methods. More importantly, it scales effectively: without modifying standard DiT training specs, solely investing more FLOPS in pretraining VTP achieves 65.8% FID improvement in downstream generation, while conventional autoencoder stagnates very early at 1/10 FLOPS. Our pre-trained models are available at https://github.com/MiniMax-AI/VTP.