mlsanityData to 14 Oct 2026

Scaling Inference-Time Search with Vision Value Model for Improved Visual Comprehension

arXiv 2412.03704 ↗First posted 4 Dec 2024cs.CV cs.CL cs.LGcited by 23

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Despite significant advancements in vision-language models (VLMs), there lacks effective approaches to enhance response quality by scaling inference-time computation. This capability is known to be a core step towards the self-improving models in recent large language model studies. In this paper, we present Vision Value Model (VisVM) that can guide VLM inference-time search to generate responses with better visual comprehension. Specifically, VisVM not only evaluates the generated sentence quality in the current search step, but also anticipates the quality of subsequent sentences that may result from the current step, thus providing a long-term value. In this way, VisVM steers VLMs away from generating sentences prone to hallucinations or insufficient detail, thereby producing higher quality responses. Experimental results demonstrate that VisVM-guided search significantly enhances VLMs' ability to generate descriptive captions with richer visual details and fewer hallucinations, compared with greedy decoding and search methods with other visual reward signals. Furthermore, we find that self-training the model with the VisVM-guided captions improve VLM's performance across a wide range of multimodal benchmarks, indicating the potential for developing self-improving VLMs. Our value model and code are available at https://github.com/si0wang/VisVM.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 55 of its 74 references that are in the map. First posted on arXiv 4 Dec 2024.

Builds on

55 of 74 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvement24 May
  2. 2024Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences19 Jan
  3. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  4. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul

Other references · 51

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  6. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  8. 2023Let's Verify Step by Stepcited by 3,789
  9. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  10. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  11. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  12. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
38 more
  1. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  2. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  3. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  4. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  5. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  6. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  7. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  8. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  9. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  10. 2021Florence: A New Foundation Model for Computer Visioncited by 804
  11. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  12. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  13. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
  14. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  15. 2018Object Hallucination in Image Captioningcited by 669
  16. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 647
  17. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  18. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 537
  19. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  20. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 344
  21. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  22. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimizationcited by 225
  23. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 198
  24. 2021Scaling Up Vision-Language Pre-training for Image Captioningcited by 196
  25. 2024Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoderscited by 139
  26. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizingcited by 125
  27. 2023Motion Planning for Autonomous Driving: The State of the Art and Future Perspectivescited by 114
  28. 2024Common 7B Language Models Already Possess Strong Math Capabilitiescited by 109
  29. 2024Calibrated Self-Rewarding Vision Language Modelscited by 78
  30. 2023VeCLIP: Improving CLIP Training via Visual-enriched Captionscited by 70
  31. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewritescited by 61
  32. 2024GenARM: Reward Guided Generation with Autoregressive Reward Model for Test-time Alignmentcited by 54
  33. 2023Mitigating Hallucination in Visual Language Models with Visual Supervisioncited by 49
  34. 2024Transfer Q Star: Principled Decoding for LLM Alignmentcited by 48
  35. 2024LiteSearch: Efficacious Tree Search for LLMcited by 31
  36. 2024Towards Self-Improvement of LLMs via MCTS: Leveraging Stepwise Knowledge with Curriculum Preference Learningcited by 30
  37. 2024LoTLIP: Improving Language-Image Pre-training for Long Text Understandingcited by 27
  38. 2023COPlanner: Plan to Roll Out Conservatively but to Explore Optimistically for Model-Based RLcited by 14

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 20; 2026: 2.

23 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  2. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvementcited by 145
  3. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scalingcited by 136
  4. 2025InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Modelcited by 74
  5. 2025LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Modelcited by 35
  6. 2025Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuningcited by 32
  7. 2025ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMscited by 25
  8. 2025Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)cited by 23
  9. 2024MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimizationcited by 21
  10. 2025VLRMBench: A Comprehensive and Challenging Benchmark for Vision-Language Reward Modelscited by 19
  11. 2025Self-Generated In-Context Examples Improve LLM Agents for Sequential Decision-Making Taskscited by 19
  12. 2025Towards Visual Text Grounding of Multimodal Large Language Modelcited by 18
11 more
  1. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  2. 2025Investigating Inference-time Scaling for Chain of Multi-modal Thought: A Preliminary Studycited by 11
  3. 2025What makes Reasoning Models Different? Follow the Reasoning Leader for Efficient Decodingcited by 8
  4. 2025CaughtCheating: Is Your MLLM a Good Cheating Detective? Exploring the Boundary of Visual Perception and Reasoningcited by 4
  5. 2025Dual-Stage Value-Guided Inference with Margin-Based Reward Adjustment for Fast and Faithful VLM Captioningcited by 2
  6. 2025From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Modelscited by 1
  7. 2025Rethinking Test Time Scaling for Flow-Matching Generative Modelscited by 1
  8. 2025Lemon: A Unified and Scalable 3D Multimodal Model for Universal Spatial Understandingcited by 1
  9. 2025Mitigating Object Hallucination via Robust Local Perception Searchcited by 0
  10. 2026Saliency-Aware Multi-Route Thinking: Revisiting Vision-Language Reasoningcited by 0
  11. 2026See Fair, Speak Truth: Equitable Attention Improves Grounding and Reduces Hallucination in Vision-Language Alignmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026On Test-Time Scaling for Vision-Language Models
  2. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions
  3. 2025Limits and Gains of Test-Time Scaling in Vision-Language Reasoning
  4. 2025Training Vision-Language Process Reward Models for Test-Time Scaling in Multimodal Reasoning: Key Insights and Lessons Learned
  5. 2026Vision-DeepResearch Benchmark: Rethinking Visual and Textual Search for Multimodal Large Language Models
  6. 2026VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs
  7. 2026AVIS: Adaptive Test-Time Scaling for Vision-Language Models
  8. 2025VReST: Enhancing Reasoning in Large Vision-Language Models through Tree Search and Self-Reward Mechanism