BlueLM-2.5-3B Technical Report
On the map: LLM reasoning → Multimodal reasoning
also in 1 more
Abstract (arXiv)
We present BlueLM-2.5-3B, a compact and unified dense Multimodal Large Language Model (MLLM) designed for efficient edge-device deployment, offering strong general-purpose and reasoning capabilities. To the best of our knowledge, this is the first 3B-scale MLLM to support both thinking and non-thinking modes, while also enabling explicit control over thinking token budget. BlueLM-2.5-3B is developed through diversified data curation, key data resampling, hybrid heterogeneous reinforcement learning, and a high-performance training infrastructure. Our model achieves superior multimodal capacity while preserving competitive pure-text performance with only 2.9 billion parameters. We conduct comprehensive evaluations across a broad range of multimodal and text-only benchmarks. In thinking mode, BlueLM-2.5-3B achieves comparable performance to Qwen3-4B on text-only benchmarks, and trails the larger Kimi-VL-A3B-16B by only about 5% on average across multimodal evaluations. In non-thinking mode, it outperforms Qwen2.5-VL-3B on the majority of multimodal benchmarks. Additionally, BlueLM-2.5-3B exhibits exceptional data efficiency. All of the aforementioned performance is achieved with substantially less total training data than Qwen2.5-VL-3B and Qwen3-4B. We hope our work contributes to the advancement of high-performance, on-device MLLMs and provides meaningful insights to the research community.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 63 of its 73 references that are in the map. First posted on arXiv 8 Jul 2025.
Builds on
63 of 73 references.
Closest ancestors · 4
Same direction, newest first
- 2025Kimi-VL Technical Report10 Apr
- 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
- 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
Other references · 59
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3 Technical Reportcited by 8,820
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2024GPT-4o System Cardcited by 4,588
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2023Let's Verify Step by Stepcited by 3,789
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
38 more
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
- 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
- 2016A Diagram Is Worth A Dozen Imagescited by 804
- 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
- 2024Phi-4 Technical Reportcited by 711
- 2024PaliGemma: A versatile 3B VLM for transfercited by 696
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
- 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
- 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
- 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
- 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
- 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 461
- 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 461
- 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 457
- 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 427
- 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
- 2023Data Selection for Language Models via Importance Resamplingcited by 329
- 2018An Empirical Model of Large-Batch Trainingcited by 281
- 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 250
- 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 177
- 2024Compact Language Models via Pruning and Knowledge Distillationcited by 148
50 most cited of 59 All 59 (JSON)
Built on by
Citing papers per year: 2025: 4; 2026: 1.
5 papers.
- 2025See Once, Then Act: Vision-Language-Action Model with Task Learning from One-Shot Video Demonstrationscited by 6
- 2025AndesVL Technical Report: An Efficient Mobile-side Multimodal Large Language Modelcited by 4
- 2025EdgeInfinite-Instruct: Bridging SFT-Based Optimization and NPU-Level Efficiency for Edge Devicescited by 1
- 2025LaV-CoT: Language-Aware Visual CoT with Multi-Aspect Reward Optimization for Real-World Multilingual VQAcited by 1
- 2026Caption First, VQA Second: Knowledge Density, Not Task Format, Drives Multimodal Scalingcited by 1
- 2026Caption First, VQA Second: Knowledge Density, Not Task Format, Drives Multimodal Scaling17 Mar 2026
- 2025See Once, Then Act: Vision-Language-Action Model with Task Learning from One-Shot Video Demonstrations8 Dec 2025
- 2025AndesVL Technical Report: An Efficient Mobile-side Multimodal Large Language Model13 Oct 2025
- 2025LaV-CoT: Language-Aware Visual CoT with Multi-Aspect Reward Optimization for Real-World Multilingual VQA12 Sep 2025
- 2025EdgeInfinite-Instruct: Bridging SFT-Based Optimization and NPU-Level Efficiency for Edge Devices1 Aug 2025
Similar, not cited
Close in wording, no citation either way
- 2025MindGPT-4ov: An Enhanced MLLM via a Multi-Stage Post-Training Paradigm
- 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models
- 2025Benchmarking Large and Small MLLMs
- 2025MindVL: Towards Efficient and Effective Training of Multimodal Large Language Models on Ascend NPUs
- 2024Efficient Multimodal Large Language Models: A Survey
- 2023A Survey on Multimodal Large Language Models
- 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning
- 2025Observe-R1: Unlocking Reasoning Abilities of MLLMs with Dynamic Progressive Reinforcement Learning