mlsanityData to 7 Oct 2026

LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts

arXiv 2407.04973 ↗First posted 6 Jul 2024cs.AI cs.CL cs.CVcited by 240

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

We propose LogicVista, an evaluation benchmark that assesses the integrated logical reasoning capabilities of multimodal large language models (MLLMs) in Visual contexts. Recent advancements in MLLMs have demonstrated various fascinating abilities, from crafting poetry based on an image to performing mathematical reasoning. However, there is still a lack of systematic evaluation of MLLMs' proficiency in logical reasoning tasks, which are essential for activities like navigation and puzzle-solving. Thus we evaluate general logical cognition abilities across 5 logical reasoning tasks encompassing 9 different capabilities, using a sample of 448 multiple-choice questions. Each question is annotated with the correct answer and the human-written reasoning behind the selection, enabling both open-ended and multiple-choice evaluation. A total of 8 MLLMs are comprehensively evaluated using LogicVista. Code and Data Available at https://github.com/Yijia-Xiao/LogicVista.

Key facts. Cited by 240 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 34 references that are in the map. First posted on arXiv 6 Jul 2024.

Builds on

23 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024ConTextual: Evaluating Context-Sensitive Text-Rich Visual Reasoning in Large Multimodal Models24 Jan

Other references · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  4. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  5. 2015VQA: Visual Question Answeringcited by 2,489
  6. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  7. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  8. 2019Towards VQA Models That Can Readcited by 1,696
  9. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  10. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  11. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  12. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
10 more
  1. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  2. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  3. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  4. 2018nocaps: novel object captioning at scalecited by 444
  5. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 398
  6. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 245
  7. 2023An Empirical Study of the Non-determinism of ChatGPT in Code Generationcited by 134
  8. 2023VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Usecited by 100
  9. 2020TAP: Text-Aware Pre-training for Text-VQA and Text-Captioncited by 95
  10. 2024MM-Soc: Benchmarking Multimodal Large Language Models in Social Media Platformscited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 106; 2026: 125.

240 papers.

Sort:

By area:

  1. 2025Qwen3-VL Technical Reportcited by 2,534
  2. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  3. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  4. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 443
  5. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 398
  6. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimizationcited by 238
  7. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 209
  8. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 193
  9. 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 174
  10. 2025Thyme: Think Beyond Imagescited by 160
  11. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
  12. 2025VisualPRM: An Effective Process Reward Model for Multimodal Reasoningcited by 134
38 more
  1. 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 119
  2. 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agentcited by 114
  3. 2025MiMo-VL Technical Reportcited by 106
  4. 2025Ovis2.5 Technical Reportcited by 103
  5. 2025VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Modelscited by 102
  6. 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 97
  7. 2025Perception-Aware Policy Optimization for Multimodal Reasoningcited by 95
  8. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  9. 2025Kwai Keye-VL 1.5 Technical Reportcited by 68
  10. 2025ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Modelscited by 56
  11. 2025Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learningcited by 51
  12. 2025Kwai Keye-VL Technical Reportcited by 47
  13. 2026Seed1.8 Model Card: Towards Generalized Real-World Agencycited by 47
  14. 2024A Survey on Evaluation of Multimodal Large Language Modelscited by 46
  15. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Modelscited by 43
  16. 2026Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipecited by 43
  17. 2025Step-GUI Technical Reportcited by 38
  18. 2025MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resourcescited by 37
  19. 2025Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Playcited by 37
  20. 2025Spotlight on Token Perception for Multimodal Reinforcement Learningcited by 37
  21. 2025OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipecited by 37
  22. 2025Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoningcited by 34
  23. 2025RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmarkcited by 33
  24. 2025VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Modelscited by 31
  25. 2026GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agentscited by 30
  26. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Modelscited by 27
  27. 2026MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methodscited by 27
  28. 2025SITE: towards Spatial Intelligence Thorough Evaluationcited by 26
  29. 2025VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoningcited by 26
  30. 2025More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Modelscited by 26
  31. 2025VOLD: Reasoning Transfer from LLMs to Vision-Language Models via On-Policy Distillationcited by 26
  32. 2025V-Thinker: Interactive Thinking with Imagescited by 26
  33. 2025Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)cited by 23
  34. 2026InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editingcited by 23
  35. 2025SpaCE-10: A Comprehensive Benchmark for Multimodal Large Language Models in Compositional Spatial Intelligencecited by 22
  36. 2025ShotBench: Expert-Level Cinematic Understanding in Vision-Language Modelscited by 22
  37. 2025WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learningcited by 21
  38. 2025Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal Perception and Generationcited by 21

50 most cited of 240 All 240 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models
  2. 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark
  3. 2025VisualPuzzles: Decoupling Multimodal Reasoning Evaluation from Domain Knowledge
  4. 2025Evaluating MLLMs with Multimodal Multi-image Reasoning Benchmark
  5. 2025Code-Vision: Evaluating Multimodal LLMs Logic Understanding and Code Generation Capabilities
  6. 2026Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Tasks
  7. 2025LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models
  8. 2024Understanding the Role of LLMs in Multimodal Evaluation Benchmarks