mlsanityData to 14 Oct 2026

Argus Inspection: Do Multimodal Large Language Models Possess the Eye of Panoptes?

arXiv 2506.14805 ↗First posted 3 Jun 2025cs.CV cs.AI cs.CLcited by 1

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

As Multimodal Large Language Models (MLLMs) continue to evolve, their cognitive and reasoning capabilities have seen remarkable progress. However, challenges in visual fine-grained perception and commonsense causal inference persist. This paper introduces Argus Inspection, a multimodal benchmark with two levels of difficulty, emphasizing detailed visual recognition while incorporating real-world commonsense understanding to evaluate causal reasoning abilities. Expanding on it, we present the Eye of Panoptes framework, which integrates a binary parametric Sigmoid metric with an indicator function, enabling a more holistic evaluation of MLLMs' responses in opinion-based reasoning tasks. Experiments conducted on 26 mainstream MLLMs reveal that the highest performance in visual fine-grained reasoning reaches only 0.46, highlighting considerable potential for enhancement. Our research offers valuable perspectives for the continued refinement of MLLMs.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 52 of its 80 references that are in the map. First posted on arXiv 3 Jun 2025.

Builds on

52 of 80 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thought13 Jan
  2. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts6 Jul
  3. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition7 May
  4. 2024Look Before You Decide: Prompting Active Deduction of MLLMs for Assumptive Reasoning19 Apr
  5. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning25 Mar
  6. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
  7. 2023ChartBench: A Benchmark for Complex Visual Reasoning in Charts26 Dec
  8. 2023Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings3 May
  9. 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb

Other references · 43

  1. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  3. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  4. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  5. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  7. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  8. 2019Towards VQA Models That Can Readcited by 1,718
  9. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  10. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  11. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  12. 2023A Survey on Multimodal Large Language Modelscited by 909
31 more
  1. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  2. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  3. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 627
  4. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  5. 2015Neural Module Networkscited by 376
  6. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 340
  7. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 300
  8. 2015Stacked Attention Networks for Image Question Answeringcited by 273
  9. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoningcited by 254
  10. 2015YFCC100M: The New Data in Multimedia Researchcited by 235
  11. 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMscited by 234
  12. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutionscited by 144
  13. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 135
  14. 2023MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devicescited by 122
  15. 2024VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?cited by 100
  16. 2024SPIQA: A Dataset for Multimodal Question Answering on Scientific Paperscited by 75
  17. 2024TableVQA-Bench: A Visual Question Answering Benchmark on Multiple Table Domainscited by 69
  18. 2023SciGraphQA: A Large-Scale Synthetic Multi-Turn Question-Answering Dataset for Scientific Graphscited by 68
  19. 2016Dynamic Memory Networks for Visual and Textual Question Answeringcited by 62
  20. 2019AI2D-RST: A multimodal corpus of 1000 primary school science diagramscited by 60
  21. 2024Needle In A Multimodal Haystackcited by 58
  22. 2016Visual Question Answering: Datasets, Algorithms, and Future Challengescited by 56
  23. 2024Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklistcited by 48
  24. 2024SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmarkcited by 35
  25. 2023Charting New Territories: Exploring the Geographic and Geospatial Capabilities of Multimodal LLMscited by 34
  26. 2024MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Modelscited by 34
  27. 2024CMMU: A Benchmark for Chinese Multi-modal Multi-type Question Understanding and Reasoningcited by 31
  28. 2023ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Modelscited by 26
  29. 2024MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understandingcited by 24
  30. 2024On Pre-training of Multimodal Language Models Customized for Chart Understandingcited by 13
  31. 2024MindBench: A Comprehensive Benchmark for Mind Map Structure Recognition and Analysiscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models
  2. 2026MM-OPD: Towards One More Bottleneck Between Perception and Reasoning
  3. 2024Multimodal Causal Reasoning Benchmark: Challenging Vision Large Language Models to Discern Causal Links Across Modalities
  4. 2025Argus: Vision-Centric Reasoning with Grounded Chain-of-Thought
  5. 2025VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity
  6. 2024MM-InstructEval: Zero-Shot Evaluation of (Multimodal) Large Language Models on Multimodal Reasoning Tasks
  7. 2025Beyond Perception: Evaluating Abstract Visual Reasoning through Multi-Stage Task
  8. 2025Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forward