# MM-OPERA: Benchmarking Open-ended Association Reasoning for Large Vision-Language Models

arXiv: https://arxiv.org/abs/2510.26937 · HTML: https://mlsanity.com/paper/2510.26937/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 26 of its 104 references that are in the map.
- First posted on arXiv 30 Oct 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 26 of 26)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension](https://mlsanity.com/paper/2307.16125/) (2023)
- [Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations](https://mlsanity.com/paper/2312.08935/) (2023)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [Yi: Open Foundation Models by 01.AI](https://mlsanity.com/paper/2403.04652/) (2024)
- [Kimi-VL Technical Report](https://mlsanity.com/paper/2504.07491/) (2025)
- [LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts](https://mlsanity.com/paper/2407.04973/) (2024)
- [LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models](https://mlsanity.com/paper/2306.09265/) (2023)
- [InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output](https://mlsanity.com/paper/2407.03320/) (2024)
- [Evaluation and Analysis of Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2308.15126/) (2023)
- [Does Spatial Cognition Emerge in Frontier Models?](https://mlsanity.com/paper/2410.06468/) (2024)
- [ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models](https://mlsanity.com/paper/2502.09696/) (2025)
- [Position: AI Evaluation Should Learn from How We Test Humans](https://mlsanity.com/paper/2306.10512/) (2023)
- [Beyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answering](https://mlsanity.com/paper/2503.11117/) (2025)
- [Evaluating General-Purpose AI with Psychometrics](https://mlsanity.com/paper/2310.16379/) (2023)
- [Evaluating Large Language Models with Psychometrics](https://mlsanity.com/paper/2406.17675/) (2024)
- [HumanEval-V: Benchmarking High-Level Visual Reasoning with Complex Diagrams in Coding Tasks](https://mlsanity.com/paper/2410.12381/) (2024)
- [Visual Riddles: a Commonsense and World Knowledge Challenge for Large Vision and Language Models](https://mlsanity.com/paper/2407.19474/) (2024)
- [The Labyrinth of Links: Navigating the Associative Maze of Multi-modal LLMs](https://mlsanity.com/paper/2410.01417/) (2024)
- [Aligning Human and LLM Judgments: Insights from EvalAssist on Task-Specific Evaluations and AI-assisted Assessment Strategy Preferences](https://mlsanity.com/paper/2410.00873/) (2024)
- [Modeling Associative Reasoning Processes](https://mlsanity.com/paper/2201.00716/) (2022)

## Built on by (most cited first, 2 of 2)

- [From Reasoning Depth to Reasoning Breadth: Evaluating Multi-Point Associative Reasoning in Large Language Models](https://mlsanity.com/paper/2608.10444/) (2026)
- [You Only Judge Once: Multi-response Reward Modeling in a Single Forward Pass](https://mlsanity.com/paper/2604.10966/) (2026)

## Abstract (arXiv)

Large Vision-Language Models (LVLMs) have exhibited remarkable progress. However, deficiencies remain compared to human intelligence, such as hallucination and shallow pattern matching. In this work, we aim to evaluate a fundamental yet underexplored intelligence: association, a cornerstone of human cognition for creative thinking and knowledge integration. Current benchmarks, often limited to closed-ended tasks, fail to capture the complexity of open-ended association reasoning vital for real-world applications. To address this, we present MM-OPERA, a systematic benchmark with 11,497 instances across two open-ended tasks: Remote-Item Association (RIA) and In-Context Association (ICA), aligning association intelligence evaluation with human psychometric principles. It challenges LVLMs to resemble the spirit of divergent thinking and convergent associative reasoning through free-form responses and explicit reasoning paths. We deploy tailored LLM-as-a-Judge strategies to evaluate open-ended outputs, applying process-reward-informed judgment to dissect reasoning with precision. Extensive empirical studies on state-of-the-art LVLMs, including sensitivity analysis of task instances, validity analysis of LLM-as-a-Judge strategies, and diversity analysis across abilities, domains, languages, cultures, etc., provide a comprehensive and nuanced understanding of the limitations of current LVLMs in associative reasoning, paving the way for more human-like and general-purpose AI. The dataset and code are available at https://github.com/MM-OPERA-Bench/MM-OPERA.