mlsanityData to 14 Oct 2026

BlueLM-2.5-3B Technical Report

arXiv 2507.05934 ↗First posted 8 Jul 2025cs.AIcited by 5

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

We present BlueLM-2.5-3B, a compact and unified dense Multimodal Large Language Model (MLLM) designed for efficient edge-device deployment, offering strong general-purpose and reasoning capabilities. To the best of our knowledge, this is the first 3B-scale MLLM to support both thinking and non-thinking modes, while also enabling explicit control over thinking token budget. BlueLM-2.5-3B is developed through diversified data curation, key data resampling, hybrid heterogeneous reinforcement learning, and a high-performance training infrastructure. Our model achieves superior multimodal capacity while preserving competitive pure-text performance with only 2.9 billion parameters. We conduct comprehensive evaluations across a broad range of multimodal and text-only benchmarks. In thinking mode, BlueLM-2.5-3B achieves comparable performance to Qwen3-4B on text-only benchmarks, and trails the larger Kimi-VL-A3B-16B by only about 5% on average across multimodal evaluations. In non-thinking mode, it outperforms Qwen2.5-VL-3B on the majority of multimodal benchmarks. Additionally, BlueLM-2.5-3B exhibits exceptional data efficiency. All of the aforementioned performance is achieved with substantially less total training data than Qwen2.5-VL-3B and Qwen3-4B. We hope our work contributes to the advancement of high-performance, on-device MLLMs and provides meaningful insights to the research community.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 63 of its 73 references that are in the map. First posted on arXiv 8 Jul 2025.

Builds on

63 of 73 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Kimi-VL Technical Report10 Apr
  2. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
  3. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  4. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul

Other references · 59

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2024GPT-4o System Cardcited by 4,588
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  10. 2023Let's Verify Step by Stepcited by 3,789
  11. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  12. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
38 more
  1. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  4. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  6. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  7. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  8. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  9. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  10. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  11. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  12. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  13. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  14. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  15. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  16. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  17. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  18. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  19. 2016A Diagram Is Worth A Dozen Imagescited by 804
  20. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  21. 2024Phi-4 Technical Reportcited by 711
  22. 2024PaliGemma: A versatile 3B VLM for transfercited by 696
  23. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  24. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  25. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
  26. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  27. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  28. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  29. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 461
  30. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 461
  31. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 457
  32. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 427
  33. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
  34. 2023Data Selection for Language Models via Importance Resamplingcited by 329
  35. 2018An Empirical Model of Large-Batch Trainingcited by 281
  36. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 250
  37. 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 177
  38. 2024Compact Language Models via Pruning and Knowledge Distillationcited by 148

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025MindGPT-4ov: An Enhanced MLLM via a Multi-Stage Post-Training Paradigm
  2. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models
  3. 2025Benchmarking Large and Small MLLMs
  4. 2025MindVL: Towards Efficient and Effective Training of Multimodal Large Language Models on Ascend NPUs
  5. 2024Efficient Multimodal Large Language Models: A Survey
  6. 2023A Survey on Multimodal Large Language Models
  7. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning
  8. 2025Observe-R1: Unlocking Reasoning Abilities of MLLMs with Dynamic Progressive Reinforcement Learning