# CRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsense

arXiv: https://arxiv.org/abs/1908.02962 · HTML: https://mlsanity.com/paper/1908.02962/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Aug 2019.

## Built on by (most cited first, 27 of 27)

- [AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learn](https://mlsanity.com/paper/2306.08640/) (2023)
- [Benchmarks for Automated Commonsense Reasoning: A Survey](https://mlsanity.com/paper/2302.04752/) (2023)
- [Recent Advances of Multimodal Continual Learning: A Comprehensive Survey](https://mlsanity.com/paper/2410.05352/) (2024)
- [Symbolic Replay: Scene Graph as Prompt for Continual Learning on VQA Task](https://mlsanity.com/paper/2208.12037/) (2022)
- [Explain Before You Answer: A Survey on Compositional Visual Reasoning](https://mlsanity.com/paper/2508.17298/) (2025)
- [From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities](https://mlsanity.com/paper/2311.00308/) (2023)
- [Robust Visual Question Answering: Datasets, Methods, and Future Challenges](https://mlsanity.com/paper/2307.11471/) (2023)
- [A survey on knowledge-enhanced multimodal learning](https://mlsanity.com/paper/2211.12328/) (2022)
- [3D Question Answering for City Scene Understanding](https://mlsanity.com/paper/2407.17398/) (2024)
- [K-VQG: Knowledge-aware Visual Question Generation for Common-sense Acquisition](https://mlsanity.com/paper/2203.07890/) (2022)
- [Vision-Language Navigation with Embodied Intelligence: A Survey](https://mlsanity.com/paper/2402.14304/) (2024)
- [ReasonVQA: A Multi-hop Reasoning Benchmark with Structural Knowledge for Visual Question Answering](https://mlsanity.com/paper/2507.16403/) (2025)
- [One VLM to Keep it Learning: Generation and Balancing for Data-free Continual Visual Question Answering](https://mlsanity.com/paper/2411.02210/) (2024)
- [Evaluating Vision-Language Models as Evaluators in Path Planning](https://mlsanity.com/paper/2411.18711/) (2024)
- [SADL: An Effective In-Context Learning Method for Compositional Visual QA](https://mlsanity.com/paper/2407.01983/) (2024)
- [Beyond Perception: Evaluating Abstract Visual Reasoning through Multi-Stage Task](https://mlsanity.com/paper/2505.21850/) (2025)
- [EarthVL: A Progressive Earth Vision-Language Understanding and Generation Framework](https://mlsanity.com/paper/2601.02783/) (2026)
- [Visually Interpretable Subtask Reasoning for Visual Question Answering](https://mlsanity.com/paper/2505.08084/) (2025)
- [OWLViz: An Open-World Benchmark for Visual Question Answering](https://mlsanity.com/paper/2503.07631/) (2025)
- [Learning by Asking Questions for Knowledge-based Novel Object Recognition](https://mlsanity.com/paper/2210.05879/) (2022)
- [Precision Empowers, Excess Distracts: Visual Question Answering With Dynamically Infused Knowledge In Language Models](https://mlsanity.com/paper/2406.09994/) (2024)
- [Questions beyond Pixels: Integrating Commonsense Knowledge in Visual Question Generation for Remote Sensing](https://mlsanity.com/paper/2602.19217/) (2026)
- [Dynamic MOdularized Reasoning for Compositional Structured Explanation Generation](https://mlsanity.com/paper/2309.07624/) (2023)
- [Evaluating Attribute Comprehension in Large Vision-Language Models](https://mlsanity.com/paper/2408.13898/) (2024)
- [Augmented Vision-Language Models: A Systematic Review](https://mlsanity.com/paper/2507.22933/) (2025)
- [Knowledge-aware Visual Question Generation for Remote Sensing Images](https://mlsanity.com/paper/2602.19224/) (2026)
- [NLKI: A lightweight Natural Language Knowledge Integration Framework for Improving Small VLMs in Commonsense VQA Tasks](https://mlsanity.com/paper/2508.19724/) (2025)

## Abstract (arXiv)

Alternatively inferring on the visual facts and commonsense is fundamental for an advanced VQA system. This ability requires models to go beyond the literal understanding of commonsense. The system should not just treat objects as the entrance to query background knowledge, but fully ground commonsense to the visual world and imagine the possible relationships between objects, e.g., "fork, can lift, food". To comprehensively evaluate such abilities, we propose a VQA benchmark, CRIC, which introduces new types of questions about Compositional Reasoning on vIsion and Commonsense, and an evaluation metric integrating the correctness of answering and commonsense grounding. To collect such questions and rich additional annotations to support the metric, we also propose an automatic algorithm to generate question samples from the scene graph associated with the images and the relevant knowledge graph. We further analyze several representative types of VQA models on the CRIC dataset. Experimental results show that grounding the commonsense to the image region and joint reasoning on vision and commonsense are still challenging for current approaches. The dataset is available at https://cricvqa.github.io.