# Liquid: Language Models are Scalable and Unified Multi-modal Generators

arXiv: https://arxiv.org/abs/2412.04332 · HTML: https://mlsanity.com/paper/2412.04332/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- Builds on 66 of its 88 references that are in the map.
- First posted on arXiv 5 Dec 2024.

## Builds on (most cited first, 30 of 66)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)

## Built on by (most cited first, 30 of 63)

- [UniTok: A Unified Tokenizer for Visual Generation and Understanding](https://mlsanity.com/paper/2502.20321/) (2025)
- [LongCat-Image Technical Report](https://mlsanity.com/paper/2512.07584/) (2025)
- [Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understanding](https://mlsanity.com/paper/2510.06308/) (2025)
- [InfinityStar: Unified Spacetime AutoRegressive Modeling for Visual Generation](https://mlsanity.com/paper/2511.04675/) (2025)
- [VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Model](https://mlsanity.com/paper/2501.12327/) (2025)
- [DualToken: Towards Unifying Visual Understanding and Generation with Dual Visual Vocabularies](https://mlsanity.com/paper/2503.14324/) (2025)
- [Omni-Video: Democratizing Unified Video Understanding and Generation](https://mlsanity.com/paper/2507.06119/) (2025)
- [Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoning](https://mlsanity.com/paper/2505.07538/) (2025)
- [RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmark](https://mlsanity.com/paper/2509.24897/) (2025)
- [UniFlow: A Unified Pixel Flow Tokenizer for Visual Understanding and Generation](https://mlsanity.com/paper/2510.10575/) (2025)
- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)
- [Uni-X: Mitigating Modality Conflict with a Two-End-Separated Architecture for Unified Multimodal Models](https://mlsanity.com/paper/2509.24365/) (2025)
- [Omni-View: Unlocking How Generation Facilitates Understanding in Unified 3D Model based on Multiview images](https://mlsanity.com/paper/2511.07222/) (2025)
- [AutoNeural: Co-Designing Vision-Language Models for NPU Inference](https://mlsanity.com/paper/2512.02924/) (2025)
- [Cross-Task Generalization Between Understanding and Generation in Unified Vision-Language Models: A Controlled Study](https://mlsanity.com/paper/2505.23043/) (2025)
- [GIR-Bench: Versatile Benchmark for Generating Images with Reasoning](https://mlsanity.com/paper/2510.11026/) (2025)
- [NextFlow: Unified Sequential Modeling Activates Multimodal Understanding and Generation](https://mlsanity.com/paper/2601.02204/) (2026)
- [Turning Internal Gap into Self-Improvement: Promoting the Generation-Understanding Unification in MLLMs](https://mlsanity.com/paper/2507.16663/) (2025)
- [UALM: Unified Audio Language Model for Understanding, Generation and Reasoning](https://mlsanity.com/paper/2510.12000/) (2025)
- [Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forward](https://mlsanity.com/paper/2511.20561/) (2025)
- [UniGame: Turning a Unified Multimodal Model Into Its Own Adversary](https://mlsanity.com/paper/2511.19413/) (2025)
- [COOPER: A Unified Model for Cooperative Perception and Reasoning in Spatial Intelligence](https://mlsanity.com/paper/2512.04563/) (2025)
- [Unified Thinker: A General Reasoning Modular Core for Image Generation](https://mlsanity.com/paper/2601.03127/) (2026)
- [Self-Improving World Modelling with Latent Actions](https://mlsanity.com/paper/2602.06130/) (2026)
- [UniMoD: Efficient Unified Multimodal Transformers with Mixture-of-Depths](https://mlsanity.com/paper/2502.06474/) (2025)
- [ProtTeX: Structure-In-Context Reasoning and Editing of Proteins with Large Language Models](https://mlsanity.com/paper/2503.08179/) (2025)
- [Growing Visual Generative Capacity for Pre-Trained MLLMs](https://mlsanity.com/paper/2510.01546/) (2025)
- [UniDDT: Unifying Multimodal Understanding and Generation with Decoupled Diffusion Transformer](https://mlsanity.com/paper/2606.16255/) (2026)
- [From Prediction to Perfection: Introducing Refinement to Autoregressive Image Generation](https://mlsanity.com/paper/2505.16324/) (2025)
- [A High-Quality Dataset and Reliable Evaluation for Interleaved Image-Text Generation](https://mlsanity.com/paper/2506.09427/) (2025)

## Abstract (arXiv)

We present Liquid, an auto-regressive generation paradigm that seamlessly integrates visual comprehension and generation by tokenizing images into discrete codes and learning these code embeddings alongside text tokens within a shared feature space for both vision and language. Unlike previous multimodal large language model (MLLM), Liquid achieves this integration using a single large language model (LLM), eliminating the need for external pretrained visual embeddings such as CLIP. For the first time, Liquid uncovers a scaling law that performance drop unavoidably brought by the unified training of visual and language tasks diminishes as the model size increases. Furthermore, the unified token space enables visual generation and comprehension tasks to mutually enhance each other, effectively removing the typical interference seen in earlier models. We show that existing LLMs can serve as strong foundations for Liquid, saving 100x in training costs while outperforming Chameleon in multimodal capabilities and maintaining language performance comparable to mainstream LLMs like LLAMA2. Liquid also outperforms models like SD v2.1 and SD-XL (FID of 5.47 on MJHQ-30K), excelling in both vision-language and text-only tasks. This work demonstrates that LLMs such as Qwen2.5 and GEMMA2 are powerful multimodal generators, offering a scalable solution for enhancing both vision-language understanding and generation. The code and models will be released at https://github.com/FoundationVision/Liquid.