# Flow to the Mode: Mode-Seeking Diffusion Autoencoders for State-of-the-Art Image Tokenization

arXiv: https://arxiv.org/abs/2503.11056 · HTML: https://mlsanity.com/paper/2503.11056/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 58 of its 68 references that are in the map.
- First posted on arXiv 14 Mar 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 58)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Training Diffusion Models with Reinforcement Learning](https://mlsanity.com/paper/2305.13301/) (2023)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation](https://mlsanity.com/paper/2406.06525/) (2024)
- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation](https://mlsanity.com/paper/2310.05737/) (2023)
- [VideoPoet: A Large Language Model for Zero-Shot Video Generation](https://mlsanity.com/paper/2312.14125/) (2023)
- [DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models](https://mlsanity.com/paper/2305.16381/) (2023)

## Built on by (most cited first, 30 of 36)

- [Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generation](https://mlsanity.com/paper/2602.11401/) (2026)
- [Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoning](https://mlsanity.com/paper/2505.07538/) (2025)
- [FASTer: Toward Efficient Autoregressive Vision Language Action Modeling via Neural Action Tokenization](https://mlsanity.com/paper/2512.04952/) (2025)
- [Single-pass Adaptive Image Tokenization for Minimum Program Search](https://mlsanity.com/paper/2507.07995/) (2025)
- [Toward Diffusible High-Dimensional Latent Spaces: A Frequency Perspective](https://mlsanity.com/paper/2511.22249/) (2025)
- [D-AR: Diffusion via Autoregressive Models](https://mlsanity.com/paper/2505.23660/) (2025)
- [MotionStrata: Hierarchical Motion Latents for Compact Video Autoencoding](https://mlsanity.com/paper/2506.07136/) (2025)
- [WeTok: Powerful Discrete Tokenization for High-Fidelity Visual Reconstruction](https://mlsanity.com/paper/2508.05599/) (2025)
- [Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey](https://mlsanity.com/paper/2507.22920/) (2025)
- [InfoTok: Adaptive Discrete Video Tokenizer via Information-Theoretic Compression](https://mlsanity.com/paper/2512.16975/) (2025)
- [TVC: Tokenized Video Compression with Ultra-Low Bit Rate](https://mlsanity.com/paper/2504.16953/) (2025)
- [SSDD: Single-Step Diffusion Decoder for Efficient Image Tokenization](https://mlsanity.com/paper/2510.04961/) (2025)
- [VA-π: Variational Policy Alignment for Pixel-Aware Autoregressive Generation](https://mlsanity.com/paper/2512.19680/) (2025)
- [VTBench: Evaluating Visual Tokenizers for Autoregressive Image Generation](https://mlsanity.com/paper/2505.13439/) (2025)
- [Scaling Diffusion Transformers Efficiently via μP](https://mlsanity.com/paper/2505.15270/) (2025)
- [REAR: Rethinking Visual Autoregressive Models via Generator-Tokenizer Consistency Regularization](https://mlsanity.com/paper/2510.04450/) (2025)
- [DINO-Tok: Adapting DINO for Visual Tokenizers](https://mlsanity.com/paper/2511.20565/) (2025)
- [CaTok: Taming Mean Flows for One-Dimensional Causal Image Tokenization](https://mlsanity.com/paper/2603.06449/) (2026)
- [Flow Autoencoders are Effective Protein Tokenizers](https://mlsanity.com/paper/2510.00351/) (2025)
- [Diffusion As Self-Distillation: End-to-End Latent Diffusion In One Model](https://mlsanity.com/paper/2511.14716/) (2025)
- [Adaptive 1D Video Diffusion Autoencoder](https://mlsanity.com/paper/2602.04220/) (2026)
- [Adaptive Protein Tokenization](https://mlsanity.com/paper/2602.06418/) (2026)
- [Semantic One-Dimensional Tokenizer for Image Reconstruction and Generation](https://mlsanity.com/paper/2603.16373/) (2026)
- [MRT: Learning Compact Representations with Mixed RWKV-Transformer for Extreme Image Compression](https://mlsanity.com/paper/2511.06717/) (2025)
- [Optimizing Input of Denoising Score Matching is Biased Towards Higher Score Norm](https://mlsanity.com/paper/2511.11727/) (2025)
- [Training-Free Vector Quantization via Gaussian VAEs](https://mlsanity.com/paper/2512.06609/) (2025)
- [VLIC: Vision-Language Models As Perceptual Judges for Human-Aligned Image Compression](https://mlsanity.com/paper/2512.15701/) (2025)
- [KVAE: Family of Tokenizers for Multimodal Generative Models](https://mlsanity.com/paper/2608.05798/) (2026)
- [Flow-Matching-Based Protein Structure Tokenizer Made Efficient and Easy](https://mlsanity.com/paper/2609.33129/) (2026)
- [UniWeTok: An Unified Binary Tokenizer with Codebook Size mathit2128 for Unified Multimodal Large Language Model](https://mlsanity.com/paper/2602.14178/) (2026)

## Abstract (arXiv)

Since the advent of popular visual generation frameworks like VQGAN and latent diffusion models, state-of-the-art image generation systems have generally been two-stage systems that first tokenize or compress visual data into a lower-dimensional latent space before learning a generative model. Tokenizer training typically follows a standard recipe in which images are compressed and reconstructed subject to a combination of MSE, perceptual, and adversarial losses. Diffusion autoencoders have been proposed in prior work as a way to learn end-to-end perceptually-oriented image compression, but have not yet shown state-of-the-art performance on the competitive task of ImageNet-1K reconstruction. We propose FlowMo, a transformer-based diffusion autoencoder that achieves a new state-of-the-art for image tokenization at multiple compression rates without using convolutions, adversarial losses, spatially-aligned two-dimensional latent codes, or distilling from other tokenizers. Our key insight is that FlowMo training should be broken into a mode-matching pre-training stage and a mode-seeking post-training stage. In addition, we conduct extensive analyses and explore the training of generative models atop the FlowMo tokenizer. Our code and models will be available at http://kylesargent.github.io/flowmo .