mlsanityData to 14 Oct 2026

MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe

arXiv 2509.18154 ↗First posted 16 Sep 2025cs.LG cs.CVcited by 186

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) are undergoing rapid progress and represent the frontier of AI development. However, their training and inference efficiency have emerged as a core bottleneck in making MLLMs more accessible and scalable. To address the challenges, we present MiniCPM-V 4.5, an 8B parameter model designed for high efficiency and strong performance. We introduce three core improvements in model architecture, data strategy and training method: a unified 3D-Resampler model architecture for highly compact encoding over images and videos, a unified learning paradigm for document knowledge and text recognition without heavy data engineering, and a hybrid reinforcement learning strategy for proficiency in both short and long reasoning modes. Comprehensive experimental results in OpenCompass evaluation show that MiniCPM-V 4.5 surpasses widely used proprietary models such as GPT-4o-latest, and significantly larger open-source models such as Qwen2.5-VL 72B. Notably, the strong performance is achieved with remarkable efficiency. For example, on the widely adopted VideoMME benchmark, MiniCPM-V 4.5 achieves state-of-the-art performance among models under 30B size, using just 46.7% GPU memory cost and 8.7% inference time of Qwen2.5-VL 7B.

Key facts. Cited by 186 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 49 of its 60 references that are in the map. First posted on arXiv 16 Sep 2025.

Builds on

49 of 60 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025MM-IFEngine: Towards Multimodal Instruction Following10 Apr
  2. 2025VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Search13 Mar
  3. 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark9 Jan
  4. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phone3 Aug
  5. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts6 Jul
  6. 2024MINT-1T: Scaling Open-Source Multimodal Data by 10x: A Multimodal Dataset with One Trillion Tokens17 Jun
  7. 2024MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI24 Apr
  8. 2023Aligning Large Multimodal Models with Factually Augmented RLHF25 Sep
  9. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities4 Aug
  10. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul

Other references · 39

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  5. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  6. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  7. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  8. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  9. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  10. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  11. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  12. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
27 more
  1. 2016A Diagram Is Worth A Dozen Imagescited by 804
  2. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  3. 2018Object Hallucination in Image Captioningcited by 669
  4. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
  5. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
  6. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 515
  7. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  8. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
  9. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 428
  10. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 410
  11. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 408
  12. 2016Synthetic Data for Text Localisation in Natural Imagescited by 396
  13. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 350
  14. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
  15. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 340
  16. 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 304
  17. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 292
  18. 2024LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Imagescited by 195
  19. 2024Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Modelscited by 139
  20. 2025Ovis2.5 Technical Reportcited by 108
  21. 2023CapsFusion: Rethinking Image-Text Data at Scalecited by 90
  22. 2020INFOTABS: Inference on Tables as Semi-structured Datacited by 86
  23. 2024Vript: A Video Is Worth Thousands of Wordscited by 71
  24. 2024OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Textcited by 59
  25. 2025FAVOR-Bench: A Comprehensive Benchmark for Fine-Grained Video Motion Understandingcited by 24
  26. 2025MulDimIF: A Multi-Dimensional Constraint Framework for Evaluating and Improving Instruction Following in Large Language Modelscited by 15
  27. 2025Flash-VL 2B: Optimizing Vision-Language Model Performance for Ultra-Low Latency and High Throughputcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 21; 2026: 165.

186 papers.

Sort:

By area:

  1. 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 139
  2. 2026Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillationcited by 77
  3. 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perceptioncited by 50
  4. 2025AutoNeural: Co-Designing Vision-Language Models for NPU Inferencecited by 17
  5. 2025Multi-Crit: Benchmarking Multimodal Judges on Pluralistic Criteria-Followingcited by 15
  6. 2026MolmoPoint: Better Pointing for VLMs with Grounding Tokenscited by 15
  7. 2026HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Bettercited by 14
  8. 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoningcited by 14
  9. 2025IWR-Bench: Can LVLMs reconstruct interactive webpage from a user interaction video?cited by 12
  10. 2025StreamEQA: Towards Streaming Video Understanding for Embodied Scenarioscited by 12
  11. 2026TimeBlind: A Spatio-Temporal Compositionality Benchmark for Video LLMscited by 11
  12. 2025MM-UAVBench: How Well Do Multimodal Large Language Models See, Think, and Plan in Low-Altitude UAV Scenarios?cited by 10
38 more
  1. 2026PhoStream: Benchmarking Real-World Streaming for Omnimodal Assistants in Mobile Scenarioscited by 10
  2. 2026Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Modelscited by 10
  3. 2026OmniCT: Towards a Unified Slice-Volume LVLM for Comprehensive CT Analysiscited by 10
  4. 2026ICON: Indirect Prompt Injection Defense for Agents based on Inference-Time Correctioncited by 10
  5. 2026VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understandingcited by 10
  6. 2025GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluationcited by 9
  7. 2026ECG-R1: Protocol-Guided and Modality-Agnostic MLLM for Reliable ECG Interpretationcited by 9
  8. 2026PEARL: Personalized Streaming Video Understanding Modelcited by 9
  9. 2026Kairos: A Regret-Aware Native World-Action Model Stack for Physical AIcited by 9
  10. 2026VISTA-Bench: Do Vision-Language Models Really Understand Visualized Text as Well as Pure Text?cited by 8
  11. 2026DART: Draft-Agreement Routing for Training-Free Adaptive Thinking Budgets in Hybrid Reasoning Modelscited by 8
  12. 2026OPD-V: Visual On-Policy Self-Distillation with Modality Balancecited by 8
  13. 2025UniComp: Rethinking Video Compression Through Informational Uniquenesscited by 7
  14. 2025VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding Taskscited by 7
  15. 2026Penguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoderscited by 7
  16. 2026Ego-Grounding for Personalized Question-Answering in Egocentric Videoscited by 7
  17. 2026Kwai Summary Attention Technical Reportcited by 7
  18. 2026Toward Native Multimodal Modeling: A Roadmapcited by 7
  19. 2026AdaCodec: A Predictive Visual Code for Video MLLMscited by 7
  20. 2025LoC-Path: Learning to Compress for Pathology Multimodal Large Language Modelscited by 6
  21. 2026SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Modelscited by 6
  22. 2026Unified Spatiotemporal Token Compression for Video-LLMs at Ultra-Low Retentioncited by 6
  23. 2026OmniScript: Towards Audio-Visual Script Generation for Long-Form Cinematic Videocited by 6
  24. 2025MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Modelscited by 5
  25. 2025Know-Show: Benchmarking Video-Language Models on Spatio-Temporal Grounded Reasoningcited by 5
  26. 2026RooflineBench: A Benchmarking Framework for On-Device LLMs via Roofline Analysiscited by 5
  27. 2026EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistantcited by 5
  28. 2026Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characterscited by 5
  29. 2026InternVideo3: Agentify Foundation Models with Multimodal Contextual Reasoningcited by 5
  30. 2026OvisOCR2 Technical Reportcited by 5
  31. 2025UniECG: Understanding and Generating ECG in One Unified Modelcited by 4
  32. 2025MM-HELIX: Boosting Multimodal Long-Chain Reflective Reasoning with Holistic Platform and Adaptive Hybrid Policy Optimizationcited by 4
  33. 2025PRISM-Bench: A Benchmark of Puzzle-Based Visual Tasks with CoT Error Detectioncited by 4
  34. 2025HERBench: A Benchmark for Multi-Evidence Integration in Video Question Answeringcited by 4
  35. 2026MVPBench: A Multi-Video Perception Evaluation Benchmark for Multi-Modal Video Understandingcited by 4
  36. 2026How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settingscited by 4
  37. 2026SocialDirector: Training-Free Social Interaction Control for Multi-Person Video Generationcited by 4
  38. 2026Artifact-Bench: Evaluating MLLMs on Detecting and Assessing the Artifacts of AI-Generated Videoscited by 4

50 most cited of 186 All 186 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Benchmarking Large and Small MLLMs
  2. 2025MiniCPM4: Ultra-Efficient LLMs on End Devices
  3. 2025MindGPT-4ov: An Enhanced MLLM via a Multi-Stage Post-Training Paradigm
  4. 2024Surveying the MLLM Landscape: A Meta-Review of Current Surveys
  5. 2025Speed Always Wins: A Survey on Efficient Architectures for Large Language Models
  6. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Models
  7. 2024Efficient Multimodal Large Language Models: A Survey
  8. 2025Observe-R1: Unlocking Reasoning Abilities of MLLMs with Dynamic Progressive Reinforcement Learning