Do Vision-Language-Models show human-like logical problem-solving capability in point and click puzzle games?
Not on the map. Why?
Abstract (arXiv)
Vision-Language(-Action) Models (VLMs) are increasingly applied to interactive environments, yet existing benchmarks often overlook the complex physical reasoning required for point-and-click puzzle games. This paper introduces Vision-Language Against The Incredible Machine (VLATIM), a benchmark designed to evaluate human-like logical problem-solving capabilities within the classic physics puzzle game The Incredible Machine 2 (TIM). Unlike existing benchmarks, VLATIM specifically targets the critical gap between high-level logical reasoning and continuous action spaces requiring precise mouse interactions. This benchmark is structured into five progressive parts, assessing capabilities that range from basic visual grounding and domain understanding to multi-step manipulation and full puzzle solving. Our results reveal a significant disparity between reasoning and execution. While large proprietary models demonstrate superior planning abilities, they struggle with precise visual grounding. Consequently, they do not yet show human-like problem-solving capabilities.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 6 of its 28 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
6 of 28 references.
References · 6
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
- 2025VLA-0: Building State-of-the-Art VLAs with Zero Modificationcited by 49
- 2025VideoGameBench: Can Vision-Language Models complete popular video games?cited by 36
- 2025JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mousecited by 28
- 2025VGRP-Bench: Visual Grid Reasoning Puzzle Benchmark for Large Vision-Language Modelscited by 26
- 2025iVISPAR -- An Interactive Visual-Spatial Reasoning Benchmark for VLMscited by 25
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Game2Real-Bench: Do Toy Games Reveal and Transfer to Real-World Visual Reasoning?
- 2026Reasoning or Pattern Matching? Probing Large Vision-Language Models with Visual Puzzles
- 2026From Perception to Action: An Interactive Benchmark for Vision Reasoning
- 2025V-MAGE: A Game Evaluation Framework for Assessing Vision-Centric Capabilities in Multimodal Large Language Models
- 2025PuzzleJAX: A Benchmark for Reasoning and Learning
- 2024Puzzle Solving using Reasoning of Large Language Models: A Survey
- 2026P1-VL: Bridging Visual Perception and Scientific Reasoning in Physics Olympiads
- 2024ING-VP: MLLMs cannot Play Easy Vision-based Games Yet