# EasyARC: Evaluating Vision Language Models on True Visual Reasoning

arXiv: https://arxiv.org/abs/2506.11595 · HTML: https://mlsanity.com/paper/2506.11595/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 7 of its 10 references that are in the map.
- First posted on arXiv 13 Jun 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 7 of 7)

- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning](https://mlsanity.com/paper/2105.04165/) (2021)
- [GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning](https://mlsanity.com/paper/2105.14517/) (2021)
- [Combining Induction and Transduction for Abstract Reasoning](https://mlsanity.com/paper/2411.02272/) (2024)
- [H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmark](https://mlsanity.com/paper/2409.01374/) (2024)
- [Is Programming by Example solved by LLMs?](https://mlsanity.com/paper/2406.08316/) (2024)

## Built on by (most cited first, 10 of 10)

- [MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources](https://mlsanity.com/paper/2509.21268/) (2025)
- [VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoning](https://mlsanity.com/paper/2507.22607/) (2025)
- [Reasoning in Computer Vision: Taxonomy, Models, Tasks, and Methodologies](https://mlsanity.com/paper/2508.10523/) (2025)
- [Object Detection with Multimodal Large Vision-Language Models: An In-depth Review](https://mlsanity.com/paper/2508.19294/) (2025)
- [Reinforcement Learning for Large Model: A Survey](https://mlsanity.com/paper/2508.08189/) (2025)
- [Do Vision-Language Models Truly Perform Vision Reasoning? A Rigorous Study of the Modality Gap](https://mlsanity.com/paper/2604.16256/) (2026)
- [Reasoning or Pattern Matching? Probing Large Vision-Language Models with Visual Puzzles](https://mlsanity.com/paper/2601.13705/) (2026)
- [LASER: A Corrective Lens for LVLMs via Visual Attention Preservation and Sink Suppression](https://mlsanity.com/paper/2607.01707/) (2026)
- [Beyond the Linear Separability Ceiling: Aligning Representations in VLMs](https://mlsanity.com/paper/2507.07574/) (2025)
- [Can VLMs Reason Robustly? A Neuro-Symbolic Investigation](https://mlsanity.com/paper/2603.23867/) (2026)

## Abstract (arXiv)

Building on recent advances in language-based reasoning models, we explore multimodal reasoning that integrates vision and text. Existing multimodal benchmarks primarily test visual extraction combined with text-based reasoning, lacking true visual reasoning with more complex interactions between vision and language. Inspired by the ARC challenge, we introduce EasyARC, a vision-language benchmark requiring multi-image, multi-step reasoning, and self-correction. EasyARC is procedurally generated, fully verifiable, and scalable, making it ideal for reinforcement learning (RL) pipelines. The generators incorporate progressive difficulty levels, enabling structured evaluation across task types and complexities. We benchmark state-of-the-art vision-language models and analyze their failure modes. We argue that EasyARC sets a new standard for evaluating true reasoning and test-time scaling capabilities in vision-language models. We open-source our benchmark dataset and evaluation code.