mlsanityData to 14 Oct 2026

MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer

arXiv 2509.16197 ↗First posted 19 Sep 2025cs.CV cs.CL cs.LGcited by 15

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Unified multimodal Large Language Models (LLMs) that can both understand and generate visual content hold immense potential. However, existing open-source models often suffer from a performance trade-off between these capabilities. We present Manzano, a simple and scalable unified framework that substantially reduces this tension by coupling a hybrid image tokenizer with a well-curated training recipe. A single shared vision encoder feeds two lightweight adapters that produce continuous embeddings for image-to-text understanding and discrete tokens for text-to-image generation within a common semantic space. A unified autoregressive LLM predicts high-level semantics in the form of text and image tokens, with an auxiliary diffusion decoder subsequently translating the image tokens into pixels. The architecture, together with a unified training recipe over understanding and generation data, enables scalable joint learning of both capabilities. Manzano achieves state-of-the-art results among unified models, and is competitive with specialist models, particularly on text-rich evaluation. Our studies show minimal task conflicts and consistent gains from scaling model size, validating our design choice of a hybrid tokenizer.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 82 of its 91 references that are in the map. First posted on arXiv 19 Sep 2025.

Builds on

82 of 91 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations23 Jun
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  3. 2024MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning30 Sep
  4. 2024xGen-MM (BLIP-3): A Family of Open Large Multimodal Models16 Aug
  5. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phone3 Aug
  6. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs24 Jun
  7. 2024MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training14 Mar
  8. 2024Efficient Multimodal Learning from Data-centric Perspective18 Feb
  9. 2024SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models8 Feb
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks21 Dec
  11. 2023VILA: On Pre-training for Visual Language Models12 Dec
  12. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models13 Nov
3 more
  1. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension30 Jul
  2. 2023JourneyDB: A Benchmark for Generative Image Understanding3 Jul
  3. 2023OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents21 Jun

Other references · 67

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2023Visual Instruction Tuningcited by 7,869
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  11. 2022Scalable Diffusion Models with Transformerscited by 6,054
  12. 2022Flow Matching for Generative Modelingcited by 5,702
38 more
  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2017Neural Discrete Representation Learningcited by 4,712
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  5. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  9. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  10. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  11. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  12. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  13. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  14. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  15. 2019Towards VQA Models That Can Readcited by 1,718
  16. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  17. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  18. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  19. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  20. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  21. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  22. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  23. 2025Qwen-Image Technical Reportcited by 1,044
  24. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  25. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  26. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  27. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  28. 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 879
  29. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  30. 2016A Diagram Is Worth A Dozen Imagescited by 804
  31. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  32. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 737
  33. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 661
  34. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
  35. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
  36. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  37. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  38. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 440

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 11.

15 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2023Planting a SEED of Vision in Large Language Model
  2. 2025Growing Visual Generative Capacity for Pre-Trained MLLMs
  3. 2024From Pixels to Tokens: Byte-Pair Encoding on Quantized Visual Modalities
  4. 2025Unified Multimodal Understanding via Byte-Pair Visual Encoding
  5. 2026UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm
  6. 2026LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Model
  7. 2024LMFusion: Adapting Pretrained Language Models for Multimodal Generation
  8. 2025ARMOR: Empowering Multimodal Understanding Model with Interleaved Multimodal Generation Capability