mlsanityData to 14 Oct 2026

Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models

arXiv 2411.14432 ↗First posted 21 Nov 2024cs.CVcited by 116

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) demonstrate enhanced capabilities and reliability by reasoning more, evolving from Chain-of-Thought prompting to product-level solutions like OpenAI o1. Despite various efforts to improve LLM reasoning, high-quality long-chain reasoning data and optimized training pipelines still remain inadequately explored in vision-language tasks. In this paper, we present Insight-V, an early effort to 1) scalably produce long and robust reasoning data for complex multi-modal tasks, and 2) an effective training pipeline to enhance the reasoning capabilities of multi-modal large language models (MLLMs). Specifically, to create long and structured reasoning data without human labor, we design a two-step pipeline with a progressive strategy to generate sufficiently long and diverse reasoning paths and a multi-granularity assessment method to ensure data quality. We observe that directly supervising MLLMs with such long and complex reasoning data will not yield ideal reasoning ability. To tackle this problem, we design a multi-agent system consisting of a reasoning agent dedicated to performing long-chain reasoning and a summary agent trained to judge and summarize reasoning results. We further incorporate an iterative DPO algorithm to enhance the reasoning agent's generation stability and quality. Based on the popular LLaVA-NeXT model and our stronger base MLLM, we demonstrate significant performance gains across challenging multi-modal benchmarks requiring visual reasoning. Benefiting from our multi-agent system, Insight-V can also easily maintain or improve performance on perception-focused multi-modal tasks.

Key facts. Cited by 116 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 45 of its 53 references that are in the map. First posted on arXiv 21 Nov 2024.

Builds on

45 of 53 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Improve Vision Language Model Chain-of-thought Reasoning21 Oct
  2. 2024Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models19 Mar
  3. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  4. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
  5. 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb
  6. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar

Other references · 39

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  8. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  11. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  12. 2024Qwen2 Technical Reportcited by 2,120
27 more
  1. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  2. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  3. 2019Towards VQA Models That Can Readcited by 1,718
  4. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  5. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  6. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  7. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  8. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  9. 2016A Diagram Is Worth A Dozen Imagescited by 804
  10. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  11. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  12. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  13. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  14. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 357
  15. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  16. 2024What matters when building vision-language models?cited by 326
  17. 2023G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Modelcited by 210
  18. 2024LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Imagescited by 195
  19. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  20. 2024Building and better understanding vision-language models: insights and future directionscited by 173
  21. 2024Ovis: Structural Embedding Alignment for Multimodal Large Language Modelcited by 169
  22. 2024Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolutioncited by 159
  23. 2023Dolphins: Multimodal Language Model for Drivingcited by 124
  24. 2024Efficient Multimodal Learning from Data-centric Perspectivecited by 122
  25. 2024MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuningcited by 76
  26. 2023Octopus: Embodied Vision-Language Programmer from Environmental Feedbackcited by 73
  27. 2024POINTS: Improving Your Vision-language Model with Affordable Strategiescited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 78; 2026: 36.

116 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  2. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 182
  3. 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 175
  4. 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectivescited by 122
  5. 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agentcited by 119
  6. 2024ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Explorationcited by 103
  7. 2025Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Modelscited by 99
  8. 2025EgoLife: Towards Egocentric Life Assistantcited by 91
  9. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 90
  10. 2025Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learningcited by 87
  11. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentationcited by 86
  12. 2025More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Modelscited by 86
38 more
  1. 2025Ola: Pushing the Frontiers of Omni-Modal Language Modelcited by 83
  2. 2025Grounded Reinforcement Learning for Visual Reasoningcited by 83
  3. 2025Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoningcited by 68
  4. 2025Unlocking Multimodal Mathematical Reasoning via Process Reward Modelcited by 65
  5. 2025VideoChat-R1.5: Visual Test-Time Scaling to Reinforce Multimodal Reasoning by Iterative Perceptioncited by 65
  6. 2025RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems?cited by 60
  7. 2025HumanOmniV2: From Understanding to Omni-Modal Reasoning with Contextcited by 55
  8. 2025Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learningcited by 53
  9. 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learningcited by 49
  10. 2025MINT-CoT: Enabling Interleaved Visual Tokens in Mathematical Chain-of-Thought Reasoningcited by 49
  11. 2025MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoningcited by 45
  12. 2025VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learningcited by 43
  13. 2025Advancing Multimodal Reasoning via Reinforcement Learning with Cold Startcited by 41
  14. 2025VisRL: Intention-Driven Visual Perception via Reinforced Reasoningcited by 39
  15. 2025Spotlight on Token Perception for Multimodal Reinforcement Learningcited by 39
  16. 2025VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Searchcited by 35
  17. 2025MemVerse: Multimodal Memory for Lifelong Learning Agentscited by 35
  18. 2025Mitigating Visual Forgetting via Take-along Visual Conditioning for Multi-modal Long CoT Reasoningcited by 32
  19. 2025UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generationcited by 32
  20. 2025Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMscited by 32
  21. 2025SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Modelscited by 29
  22. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoningcited by 28
  23. 2025AStar: Boosting Multimodal Reasoning with Automated Structured Thinkingcited by 25
  24. 2025Towards Video Thinking Test: A Holistic Benchmark for Advanced Video Reasoning and Understandingcited by 24
  25. 2025A Survey on Agentic Multimodal Large Language Modelscited by 23
  26. 2025Mind with Eyes: from Language Reasoning to Multimodal Reasoningcited by 22
  27. 2025ShotBench: Expert-Level Cinematic Understanding in Vision-Language Modelscited by 22
  28. 2025Valley2: Exploring Multimodal Models with Scalable Vision-Language Designcited by 21
  29. 2025WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learningcited by 21
  30. 2025SparseMM: Head Sparsity Emerges from Visual Concept Responses in MLLMscited by 20
  31. 2025Play to Generalize: Learning to Reason Through Game Playcited by 16
  32. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  33. 2025Fact-R1: Towards Explainable Video Misinformation Detection with Deep Reasoningcited by 14
  34. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 14
  35. 2025Revisiting Visual Understanding in Multimodal Reasoning through a Lens of Image Perturbationcited by 14
  36. 2025High-Resolution Visual Reasoning via Multi-Turn Grounding-Based Reinforcement Learningcited by 14
  37. 2025Sherlock: Self-Correcting Reasoning in Vision-Language Modelscited by 13
  38. 2025Constructive Distortion: Improving MLLMs with Attention-Guided Image Warpingcited by 13

50 most cited of 116 All 116 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Insight-V++: Towards Advanced Long-Chain Visual Reasoning with Multimodal Large Language Models
  2. 2025Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forward
  3. 2025Be My Eyes: Extending Large Language Models to New Modalities Through Multi-Agent Collaboration
  4. 2024Advancing Large Multi-modal Models with Explicit Chain-of-Reasoning and Visual Question Generation
  5. 2024Enhancing Advanced Visual Reasoning Ability of Large Language Models
  6. 2025VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models
  7. 2026Vision-aligned Latent Reasoning for Multi-modal Large Language Model
  8. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges