mlsanityData to 14 Oct 2026

CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning

arXiv 1612.06890 ↗First posted 20 Dec 2016cs.CV cs.CL cs.LGcited by 1,457

Not on the map. Why?

Cited by 308 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

When building artificial intelligence systems that can reason and answer questions about visual data, we need diagnostic tests to analyze our progress and discover shortcomings. Existing benchmarks for visual question answering can help, but have strong biases that models can exploit to correctly answer questions without reasoning. They also conflate multiple sources of error, making it hard to pinpoint model weaknesses. We present a diagnostic dataset that tests a range of visual reasoning abilities. It contains minimal biases and has detailed annotations describing the kind of reasoning each question requires. We use this dataset to analyze a variety of modern visual reasoning systems, providing novel insights into their abilities and limitations.

Key facts. Cited by 1,457 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Dec 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 3; 2020: 1; 2021: 190; 2022: 236; 2023: 261; 2024: 253; 2025: 277; 2026: 236.

1,457 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  3. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  4. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  5. 2022Visual Prompt Tuningcited by 1,681
  6. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  7. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  8. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  9. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  10. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  11. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  12. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
38 more
  1. 2021Scaling Vision with Sparse Mixture of Expertscited by 747
  2. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  3. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
  4. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
  5. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
  6. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 649
  7. 2023DataComp: In search of the next generation of multimodal datasetscited by 644
  8. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 624
  9. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 529
  10. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 454
  11. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 449
  12. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 436
  13. 2022Kubric: A scalable dataset generatorcited by 381
  14. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 370
  15. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 367
  16. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  17. 2024What matters when building vision-language models?cited by 326
  18. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 311
  19. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 304
  20. 2022Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuningcited by 294
  21. 2022Visual Spatial Reasoningcited by 293
  22. 2022Exploring Visual Prompts for Adapting Large-Scale Modelscited by 274
  23. 2021IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoningcited by 267
  24. 2021Generating Images with Sparse Representationscited by 266
  25. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 262
  26. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
  27. 2023MultiModal-GPT: A Vision and Language Model for Dialogue with Humanscited by 255
  28. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoningcited by 254
  29. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 251
  30. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 250
  31. 2021Conditional Object-Centric Learning from Videocited by 248
  32. 2021GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoningcited by 228
  33. 2021Contrastive Learning Inverts the Data Generating Processcited by 221
  34. 2023Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMCcited by 220
  35. 2024STAR: A Benchmark for Situated Reasoning in Real-World Videoscited by 213
  36. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 212
  37. 2024NVILA: Efficient Frontier Visual Language Modelscited by 209
  38. 2021Projected GANs Converge Fastercited by 208

50 most cited of 1,457 All 1,457 (JSON)