mlsanityData to 14 Oct 2026

CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoning

arXiv 2208.05358 ↗First posted 10 Aug 2022cs.LG cs.CL cs.CVcited by 120

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

We introduce CLEVR-Math, a multi-modal math word problems dataset consisting of simple math word problems involving addition/subtraction, represented partly by a textual description and partly by an image illustrating the scenario. The text describes actions performed on the scene that is depicted in the image. Since the question posed may not be about the scene in the image, but about the state of the scene before or after the actions are applied, the solver envision or imagine the state changes due to these actions. Solving these word problems requires a combination of language, visual and mathematical reasoning. We apply state-of-the-art neural and neuro-symbolic models for visual question answering on CLEVR-Math and empirically evaluate their performances. Our results show how neither method generalise to chains of operations. We discuss the limitations of the two in addressing the task of multi-modal word problem solving.

Key facts. Cited by 120 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 22 of its 30 references that are in the map. First posted on arXiv 10 Aug 2022.

Builds on

22 of 30 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality7 Apr

Other references · 21

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2017Mask R-CNNcited by 7,277
  4. 2015VQA: Visual Question Answeringcited by 2,506
  5. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  6. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  7. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  8. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 484
  9. 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 372
  10. 2018Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understandingcited by 304
  11. 2019Action Genome: Actions as Composition of Spatio-temporal Scene Graphscited by 250
  12. 2016Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Groundingcited by 243
9 more
  1. 2020ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graphcited by 211
  2. 2020Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructionscited by 208
  3. 2016FVQA: Fact-based Visual Question Answeringcited by 181
  4. 2021Graph Learning: A Surveycited by 141
  5. 2017MUTAN: Multimodal Tucker Fusion for Visual Question Answeringcited by 137
  6. 2014A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Inputcited by 112
  7. 2020Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanationscited by 94
  8. 2015Where To Look: Focus Regions for Visual Question Answeringcited by 64
  9. 2021CLEVR_HYP: A Challenge Dataset and Baselines for Visual Question Answering with Hypothetical Actions over Imagescited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 24; 2025: 58; 2026: 33.

120 papers.

Sort:

By area:

  1. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  2. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  3. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  4. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  5. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
  6. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 454
  7. 2024What matters when building vision-language models?cited by 326
  8. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLcited by 248
  9. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimizationcited by 239
  10. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 212
  11. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  12. 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 175
38 more
  1. 2024Building and better understanding vision-language models: insights and future directionscited by 173
  2. 2024InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HDcited by 159
  3. 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMscited by 147
  4. 2024Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Modelscited by 133
  5. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scalecited by 127
  6. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 125
  7. 2025Baichuan-Omni-1.5 Technical Reportcited by 111
  8. 2023GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoningcited by 108
  9. 2025Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Modelscited by 100
  10. 2025Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Modelscited by 99
  11. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  12. 2025Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learningcited by 87
  13. 2024Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-trainingcited by 82
  14. 2025InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Modelcited by 74
  15. 2025Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Modelscited by 70
  16. 2024Vision-Language Models Can Self-Improve Reasoning via Reflectioncited by 69
  17. 2025OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoningcited by 69
  18. 2024OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Textcited by 59
  19. 2025FineVision: Open Data Is All You Needcited by 53
  20. 2024Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usagecited by 51
  21. 2025OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generationcited by 50
  22. 2025SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoningcited by 46
  23. 2025Advancing Multimodal Reasoning via Reinforcement Learning with Cold Startcited by 41
  24. 2025MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resourcescited by 39
  25. 2024MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversitycited by 38
  26. 2025Spatial457: A Diagnostic Benchmark for 6D Spatial Reasoning of Large Multimodal Modelscited by 38
  27. 2025HiDe-LLaVA: Hierarchical Decoupling for Continual Instruction Tuning of Multimodal Large Language Modelcited by 31
  28. 2026LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decodingcited by 28
  29. 2025VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoningcited by 26
  30. 2025Valley2: Exploring Multimodal Models with Scalable Vision-Language Designcited by 21
  31. 2025WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learningcited by 21
  32. 2025To Sink or Not to Sink: Visual Information Pathways in Large Vision-Language Modelscited by 20
  33. 2025M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performancecited by 19
  34. 2025From Pixels to Words -- Towards Native Vision-Language Primitives at Scalecited by 18
  35. 2024POINTS: Improving Your Vision-language Model with Affordable Strategiescited by 17
  36. 2025Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Modelscited by 17
  37. 2025Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMscited by 17
  38. 2025ViLBench: A Suite for Vision-Language Process Reward Modelingcited by 16

50 most cited of 120 All 120 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026VisTIRA: Closing the Image-Text Modality Gap in Visual Math Reasoning via Structured Tool Integration
  2. 2025MV-MATH: Evaluating Multimodal Math Reasoning in Multi-Visual Contexts
  3. 2023Techniques to Improve Neural Math Word Problem Solvers
  4. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?
  5. 2026MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval
  6. 2023Math Word Problem Solving by Generating Linguistic Variants of Problem Statements
  7. 2024Towards Learning to Reason: Comparing LLMs with Neuro-Symbolic on Arithmetic Relations in Abstract Reasoning
  8. 2022CLEVR-X: A Visual Reasoning Dataset for Natural Language Explanations