# FlowTok: Flowing Seamlessly Across Text and Image Tokens

arXiv: https://arxiv.org/abs/2503.10772 · HTML: https://mlsanity.com/paper/2503.10772/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 81 of its 95 references that are in the map.
- First posted on arXiv 13 Mar 2025.

## Builds on (most cited first, 30 of 81)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)

## Built on by (most cited first, 30 of 33)

- [VITA: Vision-to-Action Flow Matching Policy](https://mlsanity.com/paper/2507.13231/) (2025)
- [Grouping First, Attending Smartly: Training-Free Acceleration for Diffusion Transformers](https://mlsanity.com/paper/2505.14687/) (2025)
- [A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokens](https://mlsanity.com/paper/2604.04913/) (2026)
- [Deeply Supervised Flow-Based Generative Models](https://mlsanity.com/paper/2503.14494/) (2025)
- [Better Source, Better Flow: Learning Condition-Dependent Source Distribution for Flow Matching](https://mlsanity.com/paper/2602.05951/) (2026)
- [Autoregressive Image Generation with Masked Bit Modeling](https://mlsanity.com/paper/2602.09024/) (2026)
- [Exploring Cross-Modal Flows for Few-Shot Learning](https://mlsanity.com/paper/2510.14543/) (2025)
- [Frequency-Aware Flow Matching for High-Quality Image Generation](https://mlsanity.com/paper/2604.15521/) (2026)
- [SCFlow: Implicitly Learning Style and Content Disentanglement with Flow Models](https://mlsanity.com/paper/2508.03402/) (2025)
- [Is There a Better Source Distribution than Gaussian? Exploring Source Distributions for Image Flow Matching](https://mlsanity.com/paper/2512.18184/) (2025)
- [FlowCast: Trajectory Forecasting for Scalable Zero-Cost Speculative Flow Matching](https://mlsanity.com/paper/2602.01329/) (2026)
- [FastFlow: Accelerating The Generative Flow Matching Models with Bandit Inference](https://mlsanity.com/paper/2602.11105/) (2026)
- [Learning by Neighbor-Aware Semantics, Deciding by Open-form Flows: Towards Robust Zero-Shot Skeleton Action Recognition](https://mlsanity.com/paper/2511.09388/) (2025)
- [Environment-Aware Channel Inference via Cross-Modal Flow: From Multimodal Sensing to Wireless Channels](https://mlsanity.com/paper/2512.04966/) (2025)
- [RMFlow: Refined Mean Flow by a Noise-Injection Step for Multimodal Generation](https://mlsanity.com/paper/2602.00849/) (2026)
- [FlowComposer: Composable Flows for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2603.16641/) (2026)
- [Residual Decoder Adapter: ID-Preserving Tokenizer Adaption for Autoregressive Text Rendering](https://mlsanity.com/paper/2606.01911/) (2026)
- [TaskTok: Delving into Task Tokens for Task-driven Image Restoration](https://mlsanity.com/paper/2606.26615/) (2026)
- [Towards General Modality Translation with Contrastive and Predictive Latent Diffusion Bridge](https://mlsanity.com/paper/2510.20819/) (2025)
- [FlowCIR: Semantic Transport via Flow Matching for Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2607.02284/) (2026)
- [FlowBind: Efficient Any-to-Any Generation with Bidirectional Flows](https://mlsanity.com/paper/2512.15420/) (2025)
- [GRRE: Leveraging G-Channel Removed Reconstruction Error for Robust Detection of AI-Generated Images](https://mlsanity.com/paper/2601.02709/) (2026)
- [Balancing Image Compression and Generation with Bootstrapped Tokenization](https://mlsanity.com/paper/2606.05552/) (2026)
- [CrossFlowDG: Bridging the Modality Gap with Cross-modal Flow Matching for Domain Generalization](https://mlsanity.com/paper/2604.16892/) (2026)
- [FullFlow: Upgrading Text-to-Image Flow Matching Models for Bidirectional Vision--Language Generation](https://mlsanity.com/paper/2605.20316/) (2026)
- [Channel-wise Vector Quantization](https://mlsanity.com/paper/2605.26089/) (2026)
- [Generative Lane Topology Reasoning via Autoregressive Model with Geometry Prior](https://mlsanity.com/paper/2606.31814/) (2026)
- [Contrastive-Augmented Flow Matching for Style-Content Disentanglement](https://mlsanity.com/paper/2607.12404/) (2026)
- [STEREOFLOW: Progressive Stereo Matching with StereoDiT and Transition Flow Matching](https://mlsanity.com/paper/2607.19986/) (2026)
- [Keep-or-Drop? Adaptive Tokenizer for Compact Video Representation](https://mlsanity.com/paper/2608.24293/) (2026)

## Abstract (arXiv)

Bridging different modalities lies at the heart of cross-modality generation. While conventional approaches treat the text modality as a conditioning signal that gradually guides the denoising process from Gaussian noise to the target image modality, we explore a much simpler paradigm-directly evolving between text and image modalities through flow matching. This requires projecting both modalities into a shared latent space, which poses a significant challenge due to their inherently different representations: text is highly semantic and encoded as 1D tokens, whereas images are spatially redundant and represented as 2D latent embeddings. To address this, we introduce FlowTok, a minimal framework that seamlessly flows across text and images by encoding images into a compact 1D token representation. Compared to prior methods, this design reduces the latent space size by 3.3x at an image resolution of 256, eliminating the need for complex conditioning mechanisms or noise scheduling. Moreover, FlowTok naturally extends to image-to-text generation under the same formulation. With its streamlined architecture centered around compact 1D tokens, FlowTok is highly memory-efficient, requires significantly fewer training resources, and achieves much faster sampling speeds-all while delivering performance comparable to state-of-the-art models. Code is available at https://github.com/TACJu/FlowTok.