# MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal Models

arXiv: https://arxiv.org/abs/2502.00698 · HTML: https://mlsanity.com/paper/2502.00698/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 39 references that are in the map.
- First posted on arXiv 2 Feb 2025.

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [DeepSeek LLM: Scaling Open-Source Language Models with Longtermism](https://mlsanity.com/paper/2401.02954/) (2024)
- [RAVEN: A Dataset for Relational and Analogical Visual rEasoNing](https://mlsanity.com/paper/1903.02741/) (2019)
- [Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learning](https://mlsanity.com/paper/2010.15875/) (2020)
- [Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning](https://mlsanity.com/paper/2010.00763/) (2020)
- [CMMMU: A Chinese Massive Multi-discipline Multimodal Understanding Benchmark](https://mlsanity.com/paper/2401.11944/) (2024)
- [Learning to Make Analogies by Contrasting Abstract Relational Structure](https://mlsanity.com/paper/1902.00120/) (2019)
- [Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices](https://mlsanity.com/paper/2201.12382/) (2022)
- [Learning Representations that Support Extrapolation](https://mlsanity.com/paper/2007.05059/) (2020)
- [A Review of Emerging Research Directions in Abstract Visual Reasoning](https://mlsanity.com/paper/2202.10284/) (2022)
- [MARVEL: Multidimensional Abstraction and Reasoning through Visual Evaluation and Learning](https://mlsanity.com/paper/2404.13591/) (2024)
- [Link-Context Learning for Multimodal LLMs](https://mlsanity.com/paper/2308.07891/) (2023)
- [Machine Number Sense: A Dataset of Visual Arithmetic Problems for Abstract and Relational Reasoning](https://mlsanity.com/paper/2004.12193/) (2020)

## Built on by (most cited first, 30 of 30)

- [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025)
- [From System 1 to System 2: A Survey of Reasoning Large Language Models](https://mlsanity.com/paper/2502.17419/) (2025)
- [Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models](https://mlsanity.com/paper/2504.21277/) (2025)
- [MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources](https://mlsanity.com/paper/2509.21268/) (2025)
- [MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMs](https://mlsanity.com/paper/2505.21327/) (2025)
- [VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoning](https://mlsanity.com/paper/2507.22607/) (2025)
- [Mind with Eyes: from Language Reasoning to Multimodal Reasoning](https://mlsanity.com/paper/2503.18071/) (2025)
- [MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM](https://mlsanity.com/paper/2505.24238/) (2025)
- [Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models](https://mlsanity.com/paper/2505.18536/) (2025)
- [SpatialViz-Bench: A Cognitively-Grounded Benchmark for Diagnosing Spatial Visualization in MLLMs](https://mlsanity.com/paper/2507.07610/) (2025)
- [Reinforcement Learning for Large Model: A Survey](https://mlsanity.com/paper/2508.08189/) (2025)
- [Boosting Vulnerability Detection of LLMs via Curriculum Preference Optimization with Synthetic Reasoning Data](https://mlsanity.com/paper/2506.07390/) (2025)
- [MMMG: a Comprehensive and Reliable Benchmark for Multitask Multimodal Generation](https://mlsanity.com/paper/2505.17613/) (2025)
- [Learning Modal-Mixed Chain-of-Thought Reasoning with Latent Embeddings](https://mlsanity.com/paper/2602.00574/) (2026)
- [Boogu-Image-0.1: Boosting Open Agentic Multimodal Generation via Understanding under a Minimal Budget](https://mlsanity.com/paper/2607.13125/) (2026)
- [IQBench: How "Smart'' Are Vision-Language Models? A Study with Human IQ Tests](https://mlsanity.com/paper/2505.12000/) (2025)
- [LogicOCR: Do Your Large Multimodal Models Excel at Logical Reasoning on Text-Rich Images?](https://mlsanity.com/paper/2505.12307/) (2025)
- [VRIQ: Benchmarking and Analyzing Visual-Reasoning IQ of VLMs](https://mlsanity.com/paper/2602.05382/) (2026)
- [Visual Reasoning Benchmark: Evaluating Multimodal LLMs on Classroom-Authentic Visual Problems from Primary Education](https://mlsanity.com/paper/2602.12196/) (2026)
- [Enhancing Multimodal In-Context Learning via Inductive-Deductive Reasoning](https://mlsanity.com/paper/2605.02378/) (2026)
- [Human-Aligned Bench: Fine-Grained Assessment of Reasoning Ability in MLLMs vs. Humans](https://mlsanity.com/paper/2505.11141/) (2025)
- [Reasoning in machine vision by learning fast and slow thinking](https://mlsanity.com/paper/2506.22075/) (2025)
- [LASER: A Corrective Lens for LVLMs via Visual Attention Preservation and Sink Suppression](https://mlsanity.com/paper/2607.01707/) (2026)
- [On the Measure of a Model: From Intelligence to Generality](https://mlsanity.com/paper/2511.11773/) (2025)
- [The Abstraction Gap in Vision-Language Causal Reasoning](https://mlsanity.com/paper/2605.28779/) (2026)
- [StemBind: When MLLMs Get Lost Between Rules and Instances in Abstract Visual Reasoning](https://mlsanity.com/paper/2606.00148/) (2026)
- [The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes](https://mlsanity.com/paper/2606.11470/) (2026)
- [ComplexityWorld: Benchmarking Vision-Language Models on Verifiable Visual Decision Making](https://mlsanity.com/paper/2608.07584/) (2026)
- [MCD: Causal Distillation of Multimodal In-Context Learning in Large Vision-Language Models](https://mlsanity.com/paper/2609.39920/) (2026)
- [Can machines reason non-verbally?](https://mlsanity.com/paper/2610.13925/) (2026)

## Abstract (arXiv)

IQ testing has served as a foundational methodology for evaluating human cognitive capabilities, deliberately decoupling assessment from linguistic background, language proficiency, or domain-specific knowledge to isolate core competencies in abstraction and reasoning. Yet, artificial intelligence research currently lacks systematic benchmarks to quantify these critical cognitive capabilities in multimodal systems. To address this crucial gap, we propose MM-IQ, a comprehensive evaluation framework, which comprises a large-scale training set with 4,776 visual reasoning problems and 2,710 meticulously curated test items spanning 8 distinct reasoning paradigms. Through systematic evaluation of existing open-source and proprietary multimodal models, our benchmark reveals striking limitations: even state-of-the-art architectures achieve only marginally superior performance to random chance (33.17% vs. 25% baseline accuracy). This substantial performance chasm highlights the inadequacy of current multimodal models in approximating fundamental human reasoning capacities, underscoring the need for paradigm-shifting advancements to bridge this cognitive divide. Moreover, inspired by the recent surge of large reasoning models, we also release a multimodal reasoning model as the baseline that is trained via reinforcement learning with verifiable reward functions, reaching competitive performance to the state-of-the-art with a notably smaller model size.