mlsanityData to 7 Oct 2026

Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search

arXiv 2412.18319 ↗First posted 24 Dec 2024cs.CV cs.AIcited by 171

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

In this work, we aim to develop an MLLM that understands and solves questions by learning to create each intermediate step of the reasoning involved till the final answer. To this end, we propose Collective Monte Carlo Tree Search (CoMCTS), a new learning-to-reason method for MLLMs, which introduces the concept of collective learning into ``tree search'' for effective and efficient reasoning-path searching and learning. The core idea of CoMCTS is to leverage collective knowledge from multiple models to collaboratively conjecture, search and identify effective reasoning paths toward correct answers via four iterative operations including Expansion, Simulation and Error Positioning, Backpropagation, and Selection. Using CoMCTS, we construct Mulberry-260k, a multimodal dataset with a tree of rich, explicit and well-defined reasoning nodes for each question. With Mulberry-260k, we perform collective SFT to train our model, Mulberry, a series of MLLMs with o1-like step-by-step Reasoning and Reflection capabilities. Extensive experiments demonstrate the superiority of our proposed methods on various benchmarks. Code will be available at https://github.com/HJYao00/Mulberry

Key facts. Cited by 171 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 75 of its 96 references that are in the map. First posted on arXiv 24 Dec 2024.

Builds on

75 of 96 references.

References · 75

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  8. 2015VQA: Visual Question Answeringcited by 2,489
  9. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  10. 2019Towards VQA Models That Can Readcited by 1,696
  11. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  12. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
38 more
  1. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  2. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  3. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  4. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  5. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  6. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  7. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  8. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  9. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  10. 2016A Diagram Is Worth A Dozen Imagescited by 793
  11. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  12. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  13. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  14. 2017Maximum Classifier Discrepancy for Unsupervised Domain Adaptationcited by 682
  15. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  16. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 559
  17. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
  18. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  19. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
  20. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 439
  21. 2021InfographicVQAcited by 430
  22. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
  23. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 404
  24. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 376
  25. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 361
  26. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347
  27. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
  28. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 338
  29. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  30. 2024What matters when building vision-language models?cited by 321
  31. 2019PlotQA: Reasoning over Scientific Plotscited by 317
  32. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 314
  33. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 267
  34. 2021IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoningcited by 262
  35. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 238
  36. 2021Mastering Atari Games with Limited Datacited by 235
  37. 2021GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoningcited by 224
  38. 2023G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Modelcited by 208

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 130; 2026: 41.

171 papers.

Sort:

By area:

  1. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  2. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  3. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 364
  4. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  5. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 296
  6. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruningcited by 278
  7. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLcited by 244
  8. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  9. 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMscited by 145
  10. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvementcited by 144
  11. 2025MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiencycited by 123
  12. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 122
38 more
  1. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  2. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentationcited by 86
  3. 2025Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learningcited by 86
  4. 2025More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Modelscited by 84
  5. 2025Grounded Reinforcement Learning for Visual Reasoningcited by 80
  6. 2025Unlocking Multimodal Mathematical Reasoning via Process Reward Modelcited by 64
  7. 2025SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learningcited by 53
  8. 2025R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPOcited by 52
  9. 2025SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Datacited by 50
  10. 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learningcited by 46
  11. 2025SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Rewardcited by 45
  12. 2025MINT-CoT: Enabling Interleaved Visual Tokens in Mathematical Chain-of-Thought Reasoningcited by 45
  13. 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Rewardcited by 45
  14. 2025Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Taskscited by 43
  15. 2025Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoningcited by 43
  16. 2025Advancing Multimodal Reasoning via Reinforcement Learning with Cold Startcited by 41
  17. 2025VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learningcited by 41
  18. 2025DriveLMM-o1: A Step-by-Step Reasoning Dataset and Large Multimodal Model for Driving Scenario Understandingcited by 35
  19. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  20. 2025AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Drivingcited by 35
  21. 2025First SFT, Second RL, Third UPT: Continual Improving Multi-Modal LLM Reasoning via Unsupervised Post-Trainingcited by 34
  22. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  23. 2025Boosting MLLM Reasoning with Text-Debiased Hint-GRPOcited by 32
  24. 2025UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generationcited by 32
  25. 2025MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMscited by 31
  26. 2025VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Modelscited by 31
  27. 2025X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domainscited by 30
  28. 2025Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMscited by 30
  29. 2025SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Modelscited by 29
  30. 2025R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Searchcited by 29
  31. 2025VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinkingcited by 27
  32. 2025VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoningcited by 26
  33. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  34. 2025AStar: Boosting Multimodal Reasoning with Automated Structured Thinkingcited by 25
  35. 2025v1: Learning to Point Visual Tokens for Multimodal Grounded Reasoningcited by 24
  36. 2025Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)cited by 23
  37. 2025Mind with Eyes: from Language Reasoning to Multimodal Reasoningcited by 22
  38. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22

50 most cited of 171 All 171 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Search
  2. 2025AirRAG: Autonomous Strategic Planning and Reasoning Steer Retrieval Augmented Generation
  3. 2024RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement
  4. 2026Think 360deg: Evaluating the Width-centric Reasoning Capability of MLLMs Beyond Depth
  5. 2025Graph-O1 : Monte Carlo Tree Search with Reinforcement Learning for Text-Attributed Graph Reasoning
  6. 2025MCTS-KBQA: Monte Carlo Tree Search with Information Gain Rewards for Knowledge Base Question Answering
  7. 2025MITS: Enhanced Tree Search Reasoning for LLMs via Pointwise Mutual Information
  8. 2024Interpretable Contrastive Monte Carlo Tree Search Reasoning