mlsanityData to 7 Oct 2026

Pixtral 12B

arXiv 2410.07073 ↗First posted 9 Oct 2024cs.CV cs.CLcited by 174

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

We introduce Pixtral-12B, a 12--billion-parameter multimodal language model. Pixtral-12B is trained to understand both natural images and documents, achieving leading performance on various multimodal benchmarks, surpassing a number of larger models. Unlike many open-source models, Pixtral is also a cutting-edge text model for its size, and does not compromise on natural language performance to excel in multimodal tasks. Pixtral uses a new vision encoder trained from scratch, which allows it to ingest images at their natural resolution and aspect ratio. This gives users flexibility on the number of tokens used to process an image. Pixtral is also able to process any number of images in its long context window of 128K tokens. Pixtral 12B substanially outperforms other open models of similar sizes (Llama-3.2 11B & Qwen-2-VL 7B). It also outperforms much larger open models like Llama-3.2 90B while being 7x smaller. We further contribute an open-source benchmark, MM-MT-Bench, for evaluating vision-language models in practical scenarios, and provide detailed analysis and code for standardized evaluation protocols for multimodal LLMs. Pixtral-12B is released under Apache 2.0 license.

Key facts. Cited by 174 papers in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 26 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

21 of 26 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution18 Sep
  2. 2024LLaVA-OneVision: Easy Visual Task Transfer6 Aug
  3. 2023Improved Baselines with Visual Instruction Tuning5 Oct

Other references · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2024The Llama 3 Herd of Modelscited by 14,192
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  11. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  12. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
6 more
  1. 2020GLU Variants Improve Transformercited by 1,724
  2. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  3. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  4. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 226
  5. 2021Lepard: Learning partial point cloud matching in rigid and deformable scenescited by 85
  6. 2023An Inverse Scaling Law for CLIP Trainingcited by 76

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 18; 2025: 104; 2026: 52.

174 papers.

Sort:

By area:

  1. 2025Qwen3-VL Technical Reportcited by 2,534
  2. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  3. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 559
  4. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 205
  5. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 165
  6. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  7. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 95
  8. 2025Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Modelscited by 92
  9. 2026Ministral 3cited by 88
  10. 2025Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understandingcited by 80
  11. 2025Ola: Pushing the Frontiers of Omni-Modal Language Modelcited by 79
  12. 2025MM-RLHF: The Next Step Forward in Multimodal LLM Alignmentcited by 79
38 more
  1. 2025InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Modelcited by 73
  2. 2025Vision Language Models are Biasedcited by 63
  3. 2025Scaling Language-Free Visual Representation Learningcited by 62
  4. 2025ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Modelscited by 56
  5. 2024VCA: Video Curious Agent for Long Video Understandingcited by 54
  6. 2024Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototypingcited by 50
  7. 2025LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?cited by 49
  8. 2025Evaluating Newtonian Mechanics in Video Generative Models with Real Physical Systemscited by 44
  9. 2025VLM4D: Towards Spatiotemporal Awareness in Vision Language Modelscited by 43
  10. 2025DesignBench: A Comprehensive Benchmark for MLLM-based Front-end Code Generationcited by 41
  11. 2025Vision as LoRAcited by 37
  12. 2025Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMscited by 35
  13. 2025VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual Tool Selectioncited by 34
  14. 2024ChatRex: Taming Multimodal LLM for Joint Perception and Understandingcited by 30
  15. 2025MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Researchcited by 30
  16. 2025MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughlycited by 29
  17. 2025DepthLM: Metric Depth From Vision Language Modelscited by 29
  18. 2025LLaVA-MORE: A Comparative Study of LLMs and Visual Backbones for Enhanced Visual Instruction Tuningcited by 27
  19. 2024CE-CoLLM: Efficient and Adaptive Large Language Models Through Cloud-Edge Collaborationcited by 25
  20. 2025MLA-Trust: Benchmarking Trustworthiness of Multimodal LLM Agents in GUI Environmentscited by 23
  21. 2025Rex-Thinker: Grounded Object Referring via Chain-of-Thought Reasoningcited by 23
  22. 2025STAR-R1: Spatial TrAnsformation Reasoning by Reinforcing Multimodal LLMscited by 21
  23. 2025SparseMM: Head Sparsity Emerges from Visual Concept Responses in MLLMscited by 21
  24. 2025Referring to Any Personcited by 20
  25. 2024HumanEval-V: Benchmarking High-Level Visual Reasoning with Complex Diagrams in Coding Taskscited by 19
  26. 2024LiveXiv -- A Multi-Modal Live Benchmark Based on Arxiv Papers Contentcited by 17
  27. 2025Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Taskscited by 17
  28. 2024p-MoD: Building Mixture-of-Depths MLLMs via Progressive Ratio Decaycited by 16
  29. 2025Does Table Source Matter? Benchmarking and Improving Multimodal Scientific Table Understanding and Reasoningcited by 14
  30. 2025Can Multi-modal (reasoning) LLMs work as deepfake detectors?cited by 14
  31. 2025Zero-shot Benchmarking: A Framework for Flexible and Scalable Automatic Evaluation of Language Modelscited by 14
  32. 2025Generalizing from SIMPLE to HARD Visual Reasoning: Can We Mitigate Modality Imbalance in VLMs?cited by 13
  33. 2025A Token-level Text Image Foundation Model for Document Understandingcited by 13
  34. 2025LogiDynamics: Unraveling the Dynamics of Inductive, Abductive and Deductive Logical Inferences in LLM Reasoningcited by 12
  35. 2025Aligning Multimodal LLM with Human Preference: A Surveycited by 12
  36. 2025Your other Left! Vision-Language Models Fail to Identify Relative Positions in Medical Imagescited by 12
  37. 2025VLM-FO1: Bridging the Gap Between High-Level Reasoning and Fine-Grained Perception in VLMscited by 12
  38. 2024Chimera: Improving Generalist Model with Domain-Specific Expertscited by 11

50 most cited of 174 All 174 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites
  2. 2026NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference
  3. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
  4. 2023Large Multimodal Models: Notes on CVPR 2023 Tutorial
  5. 2025MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer
  6. 2023VCoder: Versatile Vision Encoders for Multimodal Large Language Models
  7. 2025Chitrarth: Bridging Vision and Language for a Billion People
  8. 2025Towards General Continuous Memory for Vision-Language Models