Phi-4-reasoning-vision-15B Technical Report
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
We present Phi-4-reasoning-vision-15B, a compact open-weight multimodal reasoning model, and share the motivations, design choices, experiments, and learnings that informed its development. Our goal is to contribute practical insight to the research community on building smaller, efficient multimodal reasoning models and to share the result of these learnings as an open-weight model that is good at common vision and language tasks and excels at scientific and mathematical reasoning and understanding user interfaces. Our contributions include demonstrating that careful architecture choices and rigorous data curation enable smaller, open-weight multimodal models to achieve competitive performance with significantly less training and inference-time compute and tokens. The most substantial improvements come from systematic filtering, error correction, and synthetic augmentation -- reinforcing that data quality remains the primary lever for model performance. Systematic ablations show that high-resolution, dynamic-resolution encoders yield consistent improvements, as accurate perception is a prerequisite for high-quality reasoning. Finally, a hybrid mix of reasoning and non-reasoning data with explicit mode tokens allows a single model to deliver fast direct answers for simpler tasks and chain-of-thought reasoning for complex problems.
Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 25 of its 40 references that are in the map. First posted on arXiv 4 Mar 2026.
Builds on
25 of 40 references.
Closest ancestors · 3
Same direction, newest first
- 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction5 Dec
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
- 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar
Other references · 22
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
- 2016A Diagram Is Worth A Dozen Imagescited by 804
- 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
- 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
- 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
- 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
- 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 513
- 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 496
- 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 490
- 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
10 more
- 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
- 2024NVILA: Efficient Frontier Visual Language Modelscited by 209
- 2021Simple data balancing achieves competitive worst-group-accuracycited by 178
- 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 167
- 2024Efficient Multimodal Learning from Data-centric Perspectivecited by 122
- 2025Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generationcited by 58
- 2024VisOnlyQA: Large Vision Language Models Still Struggle with Visual Perception of Geometric Informationcited by 39
- 2025VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Searchcited by 35
- 2022Why does Throwing Away Data Improve Worst-Group Error?cited by 22
- 2025Think in Safety: Unveiling and Mitigating Safety Alignment Collapse in Multimodal Large Reasoning Modelcited by 19
Built on by
Citing papers per year: 2026: 10.
10 papers.
- 2026LatentRouter: Can We Choose the Right Multimodal Model Before Seeing Its Answer?cited by 2
- 2026Retrieval Heads Meet Vision: Uncovering How VLMs Locate and Extract Visual Informationcited by 1
- 2026UltraVR: A Diagnostic Ultra-Resolution Image-VQA Benchmark for Evidence-Grounded Reasoningcited by 0
- 2026EinSort: Sorting is All We Need for Tensorizing LLMcited by 0
- 2026Knowing You at First Glance: Inferring Apparent Personality from Facescited by 0
- 2026SciFigPlag-Bench: A Benchmark for Provenance-Aware Scientific Figure Plagiarism Detectioncited by 0
- 2026When Safety Overrides Vision: Exploring Dynamics between Vision Influence and Safety Alignment in Vision-Language Modelscited by 0
- 2026Rho: A Foundation for Efficiently Adaptable VLA Modelscited by 0
- 2026Scenes as an Attack Surface: Uncovering Scene-Grounded Safety Failures in MLLMscited by 0
- 2026ChemExplain: A Multimodal Benchmark for Explanatory Chemical Reasoningcited by 0
- 2026ChemExplain: A Multimodal Benchmark for Explanatory Chemical Reasoning12 Oct 2026
- 2026Scenes as an Attack Surface: Uncovering Scene-Grounded Safety Failures in MLLMs8 Oct 2026
- 2026Rho: A Foundation for Efficiently Adaptable VLA Models29 Sep 2026
- 2026Retrieval Heads Meet Vision: Uncovering How VLMs Locate and Extract Visual Information27 Aug 2026
- 2026When Safety Overrides Vision: Exploring Dynamics between Vision Influence and Safety Alignment in Vision-Language Models19 Aug 2026
- 2026SciFigPlag-Bench: A Benchmark for Provenance-Aware Scientific Figure Plagiarism Detection31 Jul 2026
- 2026Knowing You at First Glance: Inferring Apparent Personality from Faces16 Jul 2026
- 2026EinSort: Sorting is All We Need for Tensorizing LLM7 Jun 2026
- 2026UltraVR: A Diagnostic Ultra-Resolution Image-VQA Benchmark for Evidence-Grounded Reasoning4 Jun 2026
- 2026LatentRouter: Can We Choose the Right Multimodal Model Before Seeing Its Answer?11 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models
- 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization
- 2025MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal Models
- 2025VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity
- 2025DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models
- 2025Coherent Multimodal Reasoning with Iterative Self-Evaluation for Vision-Language Models
- 2025Mind with Eyes: from Language Reasoning to Multimodal Reasoning
- 2025Revisiting Visual Understanding in Multimodal Reasoning through a Lens of Image Perturbation