Selective Vision is the Challenge for Visual Reasoning: A Benchmark for Visual Argument Understanding
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Visual arguments, often used in advertising or social causes, rely on images to persuade viewers to do or believe something. Understanding these arguments requires selective vision: only specific visual stimuli within an image are relevant to the argument, and relevance can only be understood within the context of a broader argumentative structure. While visual arguments are readily appreciated by human audiences, we ask: are today's AI capable of similar understanding? We present VisArgs, a dataset of 1,611 images annotated with 5,112 visual premises (with regions), 5,574 commonsense premises, and reasoning trees connecting them into structured arguments. We propose three tasks for evaluating visual argument understanding: premise localization, premise identification, and conclusion deduction. Experiments show that 1) machines struggle to capture visual cues: GPT-4-O achieved 78.5% accuracy, while humans reached 98.0%. Models also performed 19.5% worse when distinguishing between irrelevant objects within the image compared to external objects. 2) Providing relevant visual premises improved model performance significantly.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 15 of its 72 references that are in the map. First posted on arXiv 27 Jun 2024.
Builds on
15 of 72 references.
Closest ancestors · 1
Same direction, newest first
Other references · 14
- 2023GPT-4 Technical Reportcited by 18,396
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2023Mistral 7Bcited by 3,164
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
- 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
- 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
- 2023Zephyr: Direct Distillation of LM Alignmentcited by 506
- 2024What matters when building vision-language models?cited by 326
- 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 302
- 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 263
- 2024DOCCI: Descriptions of Connected and Contrasting Imagescited by 132
2 more
Built on by
Citing papers per year: 2025: 3.
3 papers.
- 2025GeoExplain: Multimodal Reasoning based on Hierarchy of Visual Information in Street Viewcited by 3
- 2025Seeing is Believing? Evaluating Vision-Language Model Susceptibility in Agent-to-Agent Multimodal Persuasioncited by 1
- 2025Do Vision-Language Models Understand Visual Persuasiveness? A Diagnosis via Visual Persuasive Factorscited by 0
- 2025Do Vision-Language Models Understand Visual Persuasiveness? A Diagnosis via Visual Persuasive Factors21 Nov 2025
- 2025Seeing is Believing? Evaluating Vision-Language Model Susceptibility in Agent-to-Agent Multimodal Persuasion26 Oct 2025
- 2025GeoExplain: Multimodal Reasoning based on Hierarchy of Visual Information in Street View19 Jun 2025
Similar, not cited
Close in wording, no citation either way
- 2025From Illusion to Intention: Visual Rationale Learning for Vision-Language Reasoning
- 2026MM-OPD: Towards One More Bottleneck Between Perception and Reasoning
- 2025A Study of Commonsense Reasoning over Visual Object Properties
- 2022Zero-shot visual reasoning through probabilistic analogical mapping
- 2026Beyond Accuracy: Evaluating Grounded Visual Evidence in Thinking with Images
- 2025Visual Graph Arena: Evaluating Visual Conceptualization of Vision and Multimodal Large Language Models
- 2025Draft and Refine with Visual Experts
- 2025VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity