mlsanityData to 7 Oct 2026

Reasoning Limitations of Multimodal Large Language Models. A Case Study of Bongard Problems

arXiv 2411.01173 ↗First posted 2 Nov 2024cs.AI cs.CV cs.LGcited by 15

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Abstract visual reasoning (AVR) involves discovering shared concepts across images through analogy, akin to solving IQ test problems. Bongard Problems (BPs) remain a key challenge in AVR, requiring both visual reasoning and verbal description. We investigate whether multimodal large language models (MLLMs) can solve BPs by formulating a set of diverse MLLM-suited solution strategies and testing 4 proprietary and 4 open-access models on 3 BP datasets featuring synthetic (classic BPs) and real-world (Bongard HOI and Bongard-OpenWorld) images. Despite some successes on real-world datasets, MLLMs struggle with synthetic BPs. To explore this gap, we introduce Bongard-RWR, a dataset representing synthetic BP concepts using real-world images. Our findings suggest that weak MLLM performance on classical BPs is not due to the domain specificity, but rather comes from their general AVR limitations. Code and dataset are available at: https://github.com/pavonism/bongard-rwr

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 91 references that are in the map. First posted on arXiv 2 Nov 2024.

Builds on

16 of 91 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Bongard in Wonderland: Visual Puzzles that Still Make AI Go Mad?25 Oct

Other references · 15

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  5. 2023Mistral 7Bcited by 3,065
  6. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  7. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  8. 2019On the Measure of Intelligencecited by 129
  9. 2024What is the Visual Cognition Gap between Humans and Multimodal LLMs?cited by 30
  10. 2018Solving Bongard Problems with a Visual Language and Pragmatic Reasoningcited by 21
  11. 2022Evaluating Understanding on Conceptual Abstraction Benchmarkscited by 13
  12. 2021Visual analogy: Deep learning versus compositional modelscited by 11
3 more
  1. 2024MindSet: Vision. A toolbox for testing DNNs on key psychological experimentscited by 8
  2. 2021Using Program Synthesis and Inductive Logic Programming to solve Bongard Problemscited by 6
  3. 2024The Cognitive Capabilities of Generative AI: A Comparative Analysis with Human Benchmarkscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 5.

15 papers.

Sort:

By area:

  1. 2025Do AI Models Perform Human-like Abstract Reasoning Across Modalities?cited by 10
  2. 2025Bongard-RWR+: Real-World Representations of Fine-Grained Concepts in Bongard Problemscited by 7
  3. 2025Synthesizing Visual Concepts as Vision-Language Programscited by 6
  4. 2025Beyond Text: Implementing Multimodal Large Language Model-Powered Multi-Agent Systems Using a No-Code Platformcited by 5
  5. 2025SIRI-Bench: Challenging VLMs' Spatial Intelligence through Complex Reasoning Taskscited by 4
  6. 2026Toward Cognitive Supersensing in Multimodal Large Language Modelcited by 4
  7. 2025Downscaling Intelligence: Exploring Perception and Reasoning Bottlenecks in Small Multimodal Modelscited by 2
  8. 2025SPHINX: A Synthetic Environment for Visual Perception and Reasoningcited by 2
  9. 2026Reasoning or Pattern Matching? Probing Large Vision-Language Models with Visual Puzzlescited by 2
  10. 2025From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Modelscited by 1
  11. 2026Playing ZendoWorld: Challenging AI Agents on Active Visual Concept Inductioncited by 1
  12. 2025Beyond the Linear Separability Ceiling: Aligning Representations in VLMscited by 0
3 more
  1. 2025Neurophysiological Characteristics of Adaptive Reasoning for Creative Problem-Solving Strategycited by 0
  2. 2026GenPlanner: From Noise to Plans -- Emergent Reasoning in Flow Matching and Diffusion Modelscited by 0
  3. 2026World-Time Compute with Verified Code World Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Bongards at the Boundary of Perception and Reasoning: Programs or Language?
  2. 2025VOILA: Evaluation of MLLMs For Perceptual Understanding and Analogical Reasoning
  3. 2025ImageNet-Think-250K: A Large-Scale Synthetic Dataset for Multimodal Reasoning for Vision Language Models
  4. 2025VisualQuest: A Benchmark for Abstract Visual Reasoning in MLLMs
  5. 2025LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models
  6. 2025On Data Synthesis and Post-training for Visual Abstract Reasoning
  7. 2025Revisiting Visual Understanding in Multimodal Reasoning through a Lens of Image Perturbation
  8. 2025VGR: Visual Grounded Reasoning