# VISREAS: Complex Visual Reasoning with Unanswerable Questions

arXiv: https://arxiv.org/abs/2403.10534 · HTML: https://mlsanity.com/paper/2403.10534/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 7 of its 40 references that are in the map.
- First posted on arXiv 23 Feb 2024.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 7 of 7)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Multimodal Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2302.00923/) (2023)
- [ViperGPT: Visual Inference via Python Execution for Reasoning](https://mlsanity.com/paper/2303.08128/) (2023)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)

## Built on by (most cited first, 9 of 9)

- [Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark](https://mlsanity.com/paper/2501.05444/) (2025)
- [Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questions](https://mlsanity.com/paper/2404.11023/) (2024)
- [Unmasking Deceptive Visuals: Benchmarking Multimodal Large Language Models on Misleading Chart Question Answering](https://mlsanity.com/paper/2503.18172/) (2025)
- [Unsolvable Problem Detection: Robust Understanding Evaluation for Large Multimodal Models](https://mlsanity.com/paper/2403.20331/) (2024)
- [SceneAlign: Aligning Multimodal Reasoning to Scene Graphs in Complex Visual Scenes](https://mlsanity.com/paper/2601.05600/) (2026)
- [PRISM-Bench: A Benchmark of Puzzle-Based Visual Tasks with CoT Error Detection](https://mlsanity.com/paper/2510.23594/) (2025)
- [CLASH: A Benchmark for Cross-Modal Contradiction Detection](https://mlsanity.com/paper/2511.19199/) (2025)
- [VisualQuest: A Benchmark for Abstract Visual Reasoning in MLLMs](https://mlsanity.com/paper/2503.19936/) (2025)
- [When Does an Image Determine the Answer? Benchmarking Visual Answerability across Charts and Scenes](https://mlsanity.com/paper/2609.34480/) (2026)

## Abstract (arXiv)

Verifying a question's validity before answering is crucial in real-world applications, where users may provide imperfect instructions. In this scenario, an ideal model should address the discrepancies in the query and convey them to the users rather than generating the best possible answer. Addressing this requirement, we introduce a new compositional visual question-answering dataset, VISREAS, that consists of answerable and unanswerable visual queries formulated by traversing and perturbing commonalities and differences among objects, attributes, and relations. VISREAS contains 2.07M semantically diverse queries generated automatically using Visual Genome scene graphs. The unique feature of this task, validating question answerability with respect to an image before answering, and the poor performance of state-of-the-art models inspired the design of a new modular baseline, LOGIC2VISION that reasons by producing and executing pseudocode without any external modules to generate the answer. LOGIC2VISION outperforms generative models in VISREAS (+4.82% over LLaVA-1.5; +12.23% over InstructBLIP) and achieves a significant gain in performance against the classification models.