mlsanityData to 14 Oct 2026

VLM2-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cues

arXiv 2502.12084 ↗First posted 17 Feb 2025cs.CLcited by 33

Not on the map. Why?

Cited by 23 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Visually linking matching cues is a crucial ability in daily life, such as identifying the same person in multiple photos based on their cues, even without knowing who they are. Despite the extensive knowledge that vision-language models (VLMs) possess, it remains largely unexplored whether they are capable of performing this fundamental task. To address this, we introduce VLM2-Bench, a benchmark designed to assess whether VLMs can Visually Link Matching cues, with 9 subtasks and over 3,000 test cases. Comprehensive evaluation across twelve VLMs, along with further analysis of various language-side and vision-side prompting methods, leads to a total of eight key findings. We identify critical challenges in models' ability to link visual cues, highlighting a significant performance gap. Based on these insights, we advocate for (i) enhancing core visual capabilities to improve adaptability and reduce reliance on prior knowledge, (ii) establishing clearer principles for integrating language-based reasoning in vision-centric tasks to prevent unnecessary biases, and (iii) shifting vision-text training paradigms toward fostering models' ability to independently structure and infer relationships among visual cues.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 36 of its 67 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

36 of 67 references.

References · 36

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  6. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  7. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  8. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  9. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  10. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  11. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  12. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
24 more
  1. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  2. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  3. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 561
  4. 2024Long Context Transfer from Language to Visioncited by 515
  5. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 368
  6. 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 363
  7. 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 188
  8. 2024OmniEdit: Building Image Editing Generalist Models Through Specialist Supervisioncited by 145
  9. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performancecited by 117
  10. 2024NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samplescited by 97
  11. 2024MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMscited by 90
  12. 2024Towards Open-ended Visual Quality Comparisoncited by 85
  13. 2023ImagenHub: Standardizing the evaluation of conditional image generation modelscited by 77
  14. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 67
  15. 2024Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Modelscited by 57
  16. 2024Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoningcited by 54
  17. 2024From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Modelscited by 53
  18. 2024Image Textualization: An Automatic Framework for Creating Accurate and Detailed Image Descriptionscited by 43
  19. 2024MIBench: Evaluating Multimodal Large Language Models over Multiple Imagescited by 39
  20. 2024Self-Correction is More than Refinement: A Learning Framework for Visual and Language Reasoning Taskscited by 32
  21. 2024HEMM: Holistic Evaluation of Multimodal Foundation Modelscited by 21
  22. 2024MACAROON: Training Vision-Language Models To Be Your Engaged Partnerscited by 10
  23. 2023SmartBook: AI-Assisted Situation Report Generation for Intelligence Analystscited by 9
  24. 2024Large Language Models are Skeptics: False Negative Problem of Input-conflicting Hallucinationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 18; 2026: 15.

33 papers.

Sort:

By area:

  1. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  2. 2025MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs for the Metaversecited by 51
  3. 2025VScan: Rethinking Visual Token Reduction for Efficient Large Vision-Language Modelscited by 41
  4. 2025Backdoor Cleaning without External Guidance in MLLM Fine-tuningcited by 35
  5. 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 29
  6. 2025A Personalized Conversational Benchmark: Towards Simulating Personalized Conversationscited by 17
  7. 2025Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMscited by 16
  8. 2026PROGRESSLM: Towards Progress Reasoning in Vision-Language Modelscited by 15
  9. 2026GEditBench v2: A Human-Aligned Benchmark for General Image Editingcited by 14
  10. 2026ApET: Approximation-Error Guided Token Compression for Efficient VLMscited by 12
  11. 2025MMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibrationcited by 11
  12. 2025MultiVerse: A Multi-Turn Conversation Benchmark for Evaluating Large Vision and Language Modelscited by 10
21 more
  1. 2025TAMP: Token-Adaptive Layerwise Pruning in Multimodal Large Language Modelscited by 9
  2. 2025MiCo: Multi-image Contrast for Reinforcement Visual Reasoningcited by 9
  3. 2025Unveiling the Lack of LVLM Robustness to Fundamental Visual Variations: Why and Path Forwardcited by 7
  4. 2025MR. Judge: Multimodal Reasoner as a Judgecited by 7
  5. 2025Rethinking Explainability in the Era of Multimodal AIcited by 6
  6. 2025A Review of 3D Object Detection with Vision-Language Modelscited by 4
  7. 2025CultureCLIP: Empowering CLIP with Cultural Awareness through Synthetic Images and Contextualized Captionscited by 3
  8. 2025MEENA (PersianMMMU): Multimodal-Multilingual Educational Exams for N-level Assessmentcited by 3
  9. 2025CrossCheck-Bench: Diagnosing Compositional Failures in Multimodal Conflict Resolutioncited by 3
  10. 2026TCAP: Tri-Component Attention Profiling for Unsupervised Backdoor Detection in MLLM Fine-Tuningcited by 3
  11. 2026ReMoT: Reinforcement Learning with Motion Contrast Tripletscited by 3
  12. 2026Self-Purification Mitigates Backdoors in Multimodal Diffusion Language Modelscited by 2
  13. 2026Looking Back and Forth: Cross-Image Attention Calibration and Attentive Preference Learning for Multi-Image Hallucination Mitigationcited by 2
  14. 2026Anatomy of a Lie: A Multi-Stage Diagnostic Framework for Tracing Hallucinations in Vision-Language Modelscited by 2
  15. 2026Seek-and-Solve: Benchmarking MLLMs for Visual Clue-Driven Reasoning in Daily Scenarioscited by 2
  16. 2026LENS: Adaptive Spatio-Temporal Zooming for Keyframe Sampling in Long-Form Videoscited by 2
  17. 2026DiffSpot: Can VLMs Spot Fine-Grained Visual Differences in Web Interfaces?cited by 1
  18. 2026MiSCHiEF: A Benchmark in Minimal-Pairs of Safety and Culture for Holistic Evaluation of Fine-Grained Image-Caption Alignmentcited by 0
  19. 2026Position: The Systemic Lack of Agency in Visual Reasoningcited by 0
  20. 2026DataClaw0: Agentic Tailoring Multimodal Data from Raw Streamscited by 0
  21. 2026Video2Skill: From Streaming Experience to Reusable Embodied Skillscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs
  2. 2026VLMs Need Words: Vision Language Models Ignore Visual Detail In Favor of Semantic Anchors
  3. 2025PairBench: Are Vision-Language Models Reliable at Comparing What They See?
  4. 2026Investigating Relational Reasoning in VLMs
  5. 2024VLM's Eye Examination: Instruct and Inspect Visual Competency of Vision Language Models
  6. 2025Mixed Signals: Decoding VLMs' Reasoning and Underlying Bias in Vision-Language Conflict
  7. 2025Hidden in plain sight: VLMs overlook their visual representations
  8. 2026VRIQ: Benchmarking and Analyzing Visual-Reasoning IQ of VLMs