mlsanityData to 7 Oct 2026

Decomposing Complex Visual Comprehension into Atomic Visual Skills for Vision Language Models

arXiv 2505.20021 ↗First posted 26 May 2025cs.CV cs.AIcited by 4

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Recent Vision-Language Models (VLMs) have demonstrated impressive multimodal comprehension and reasoning capabilities, yet they often struggle with trivially simple visual tasks. In this work, we focus on the domain of basic 2D Euclidean geometry and systematically categorize the fundamental, indivisible visual perception skills, which we refer to as atomic visual skills. We then introduce the Atomic Visual Skills Dataset (AVSD) for evaluating VLMs on the atomic visual skills. Using AVSD, we benchmark state-of-the-art VLMs and find that they struggle with these tasks, despite being trivial for adult humans. Our findings highlight the need for purpose-built datasets to train and evaluate VLMs on atomic, rather than composite, visual perception tasks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 51 of its 58 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

51 of 58 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  2. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality7 Apr
  3. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar

Other references · 48

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2021Program Synthesis with Large Language Modelscited by 3,509
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  8. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  9. 2015VQA: Visual Question Answeringcited by 2,489
  10. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  11. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  12. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
36 more
  1. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  2. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  3. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  4. 2016A Diagram Is Worth A Dozen Imagescited by 793
  5. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  6. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  7. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
  8. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  9. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  10. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
  11. 2019PlotQA: Reasoning over Scientific Plotscited by 317
  12. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 246
  13. 2023SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionalitycited by 238
  14. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 199
  15. 2023Teaching CLIP to Count to Tencited by 194
  16. 2022CREPE: Can Vision-Language Foundation Models Reason Compositionally?cited by 177
  17. 2024An Introduction to Vision-Language Modelingcited by 175
  18. 2023Physics of Language Models: Part 3.2, Knowledge Manipulationcited by 142
  19. 2022UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expressioncited by 137
  20. 2023Teaching Arithmetic to Small Transformerscited by 121
  21. 2022VL-CheckList: Evaluating Pre-trained Vision-Language Models with Objects, Attributes and Relationscited by 113
  22. 2023A Theory for Emergence of Complex Skills in Language Modelscited by 111
  23. 2023GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoningcited by 107
  24. 2022Does CLIP Bind Concepts? Probing Compositionality in Large Image Modelscited by 101
  25. 2023Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Taskcited by 91
  26. 2021Making Transformers Solve Compositional Taskscited by 72
  27. 2023Revisiting the Role of Language Priors in Vision-Language Modelscited by 50
  28. 2024Reverse Training to Nurse the Reversal Cursecited by 48
  29. 2024Multimodal Table Understandingcited by 43
  30. 2024Dual Operating Modes of In-Context Learningcited by 42
  31. 2024Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic taskscited by 40
  32. 2024Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalabilitycited by 39
  33. 2024Out-of-distribution generalization via composition: a lens through induction heads in Transformerscited by 28
  34. 2023Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Taskscited by 21
  35. 2024Can Models Learn Skill Composition from Examples?cited by 17
  36. 2024Measuring Vision-Language STEM Skills of Neural Modelscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025EncQA: Benchmarking Vision-Language Models on Visual Encodings for Chartscited by 11
  2. 2025AVA-Bench: Atomic Visual Ability Benchmark for Vision Foundation Modelscited by 5
  3. 2026RadThinking: A Dataset for Longitudinal Clinical Reasoning in Radiologycited by 4
  4. 2026CARV: A Diagnostic Benchmark for Compositional Analogical Reasoning in Multimodal LLMscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026VRIQ: Benchmarking and Analyzing Visual-Reasoning IQ of VLMs
  2. 2026The Last Visible Pixel: Probing Fine-Scale Perception in Vision-Language Models
  3. 2025Visual Language Models show widespread visual deficits on neuropsychological tests
  4. 2025VLMs have Tunnel Vision: Evaluating Nonlocal Visual Reasoning in Leading VLMs
  5. 2024MARVEL: Multidimensional Abstraction and Reasoning through Visual Evaluation and Learning
  6. 2026PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models
  7. 2026From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models
  8. 2025MathSight: A Benchmark Exploring Have Vision-Language Models Really Seen in University-Level Mathematical Reasoning?