# MM-CondChain: A Programmatically Verified Benchmark for Visually Grounded Deep Compositional Reasoning arXiv: https://arxiv.org/abs/2603.12266 ยท HTML: https://mlsanity.com/paper/2603.12266/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. - Builds on 48 of its 57 references that are in the map. - First posted on arXiv 12 Mar 2026. ## Placed in - LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/) - Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/) ## Builds on (most cited first, 30 of 48) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024) - [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025) - [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023) - [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025) - [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025) - [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023) - [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025) - [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016) - [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023) - [Phi-4 Technical Report](https://mlsanity.com/paper/2412.08905/) (2024) - [When and why vision-language models behave like bags-of-words, and what to do about it?](https://mlsanity.com/paper/2210.01936/) (2022) - [MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?](https://mlsanity.com/paper/2403.14624/) (2024) - [Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality](https://mlsanity.com/paper/2204.03162/) (2022) - [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026) - [GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning](https://mlsanity.com/paper/2507.01006/) (2025) - [Android in the Wild: A Large-Scale Dataset for Android Device Control](https://mlsanity.com/paper/2307.10088/) (2023) - [Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning](https://mlsanity.com/paper/2403.16999/) (2024) - [LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts](https://mlsanity.com/paper/2407.04973/) (2024) - [SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionality](https://mlsanity.com/paper/2306.14610/) (2023) - [RAVEN: A Dataset for Relational and Analogical Visual rEasoNing](https://mlsanity.com/paper/1903.02741/) (2019) - [Generalizing Verifiable Instruction Following](https://mlsanity.com/paper/2507.02833/) (2025) - [MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence](https://mlsanity.com/paper/2505.23764/) (2025) - [MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs](https://mlsanity.com/paper/2501.17399/) (2025) - [Android in the Zoo: Chain-of-Action-Thought for GUI Agents](https://mlsanity.com/paper/2403.02713/) (2024) - [Benchmarking Complex Instruction-Following with Multiple Constraints Composition](https://mlsanity.com/paper/2407.03978/) (2024) - [LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs](https://mlsanity.com/paper/2501.06186/) (2025) - [FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Models](https://mlsanity.com/paper/2310.20410/) (2023) - [VL-CheckList: Evaluating Pre-trained Vision-Language Models with Objects, Attributes and Relations](https://mlsanity.com/paper/2207.00221/) (2022) - [VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models](https://mlsanity.com/paper/2504.15279/) (2025) ## Built on by (most cited first, 2 of 2) - [Video-IFBench: Evaluating Instruction Following of Multimodal LLMs in Video Understanding Scenarios](https://mlsanity.com/paper/2608.25529/) (2026) - [Hob-VL: A Benchmark for Visually Grounded Boolean Reasoning](https://mlsanity.com/paper/2610.01605/) (2026) ## Abstract (arXiv) Multimodal Large Language Models (MLLMs) are increasingly used to carry out visual workflows such as navigating GUIs, where the next step depends on verified visual compositional conditions (e.g., "if a permission dialog appears and the color of the interface is green, click Allow") and the process may branch or terminate early. Yet this capability remains under-evaluated: existing benchmarks focus on shallow-compositions or independent-constraints rather than deeply chained compositional conditionals. In this paper, we introduce MM-CondChain, a benchmark for visually grounded deep compositional reasoning. Each benchmark instance is organized as a multi-layer reasoning chain, where every layer contains a non-trivial compositional condition grounded in visual evidence and built from multiple objects, attributes, or relations. To answer correctly, an MLLM must perceive the image in detail, reason over multiple visual elements at each step, and follow the resulting execution path to the final outcome. To scalably construct such workflow-style data, we propose an agentic synthesis pipeline: a Planner orchestrates layer-by-layer generation of compositional conditions, while a Verifiable Programmatic Intermediate Representation (VPIR) ensures each layer's condition is mechanically verifiable. A Composer then assembles these verified layers into complete instructions. Using this pipeline, we construct benchmarks across three visual domains: natural images, data charts, and GUI trajectories. Experiments on a range of MLLMs show that even the strongest model attains only 53.33 Path F1, with sharp drops on hard negatives and as depth or predicate complexity grows, confirming that deep compositional reasoning remains a fundamental challenge.