# VQA-LOL: Visual Question Answering under the Lens of Logic

arXiv: https://arxiv.org/abs/2002.08325 · HTML: https://mlsanity.com/paper/2002.08325/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2020.

## Built on by (most cited first, 30 of 48)

- [Vision-Language Pre-training: Basics, Recent Advances, and Future Trends](https://mlsanity.com/paper/2210.09263/) (2022)
- [Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions](https://mlsanity.com/paper/2209.03430/) (2022)
- [Benchmarking Spatial Relationships in Text-to-Image Generation](https://mlsanity.com/paper/2212.10015/) (2022)
- [Compressing Visual-linguistic Model via Knowledge Distillation](https://mlsanity.com/paper/2104.02096/) (2021)
- [Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models](https://mlsanity.com/paper/2106.00245/) (2021)
- [Human-Adversarial Visual Question Answering](https://mlsanity.com/paper/2106.02280/) (2021)
- [EHRXQA: A Multi-Modal Question Answering Dataset for Electronic Health Records with Chest X-ray Images](https://mlsanity.com/paper/2310.18652/) (2023)
- [Weakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answering](https://mlsanity.com/paper/2109.04014/) (2021)
- [CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation](https://mlsanity.com/paper/2211.00295/) (2022)
- [MultiViz: Towards Visualizing and Understanding Multimodal Models](https://mlsanity.com/paper/2207.00056/) (2022)
- [Getting it Right: Improving Spatial Consistency in Text-to-Image Models](https://mlsanity.com/paper/2404.01197/) (2024)
- [UNK-VQA: A Dataset and a Probe into the Abstention Ability of Multi-modal Large Models](https://mlsanity.com/paper/2310.10942/) (2023)
- [Robust Visual Question Answering: Datasets, Methods, and Future Challenges](https://mlsanity.com/paper/2307.11471/) (2023)
- [Dynamic Multimodal Evaluation with Flexible Complexity by Vision-Language Bootstrapping](https://mlsanity.com/paper/2410.08695/) (2024)
- [Bridging the Gap between 2D and 3D Visual Question Answering: A Fusion Approach for 3D VQA](https://mlsanity.com/paper/2402.15933/) (2024)
- [Rethinking Data Augmentation for Robust Visual Question Answering](https://mlsanity.com/paper/2207.08739/) (2022)
- [Detector-Free Weakly Supervised Grounding by Separation](https://mlsanity.com/paper/2104.09829/) (2021)
- [Measuring Compositional Consistency for Video Question Answering](https://mlsanity.com/paper/2204.07190/) (2022)
- [Discovering the Unknown Knowns: Turning Implicit Knowledge in the Dataset into Explicit Training Examples for Visual Question Answering](https://mlsanity.com/paper/2109.06122/) (2021)
- [CRIPP-VQA: Counterfactual Reasoning about Implicit Physical Properties via Video Question Answering](https://mlsanity.com/paper/2211.03779/) (2022)
- [Weakly Supervised Relative Spatial Reasoning for Visual Question Answering](https://mlsanity.com/paper/2109.01934/) (2021)
- [Exploring Question Decomposition for Zero-Shot VQA](https://mlsanity.com/paper/2310.17050/) (2023)
- [Self-Training Large Language Models for Improved Visual Program Synthesis With Visual Reinforcement](https://mlsanity.com/paper/2404.04627/) (2024)
- [Domain-robust VQA with diverse datasets and methods but no target labels](https://mlsanity.com/paper/2103.15974/) (2021)
- [RG-SAN: Rule-Guided Spatial Awareness Network for End-to-End 3D Referring Expression Segmentation](https://mlsanity.com/paper/2412.02402/) (2024)
- [Consistency-preserving Visual Question Answering in Medical Imaging](https://mlsanity.com/paper/2206.13296/) (2022)
- [Semantically Distributed Robust Optimization for Vision-and-Language Inference](https://mlsanity.com/paper/2110.07165/) (2021)
- [RoCOCO: Robustness Benchmark of MS-COCO to Stress-test Image-Text Matching Models](https://mlsanity.com/paper/2304.10727/) (2023)
- [Unsupervised Natural Language Inference Using PHL Triplet Generation](https://mlsanity.com/paper/2110.08438/) (2021)
- [MLP Architectures for Vision-and-Language Modeling: An Empirical Study](https://mlsanity.com/paper/2112.04453/) (2021)

## Abstract (arXiv)

Logical connectives and their implications on the meaning of a natural language sentence are a fundamental aspect of understanding. In this paper, we investigate whether visual question answering (VQA) systems trained to answer a question about an image, are able to answer the logical composition of multiple such questions. When put under this textitLens of Logic, state-of-the-art VQA models have difficulty in correctly answering these logically composed questions. We construct an augmentation of the VQA dataset as a benchmark, with questions containing logical compositions and linguistic transformations (negation, disjunction, conjunction, and antonyms). We propose our Lens of Logic (LOL) model which uses question-attention and logic-attention to understand logical connectives in the question, and a novel Fr'echet-Compatibility Loss, which ensures that the answers of the component questions and the composed question are consistent with the inferred logical operation. Our model shows substantial improvement in learning logical compositions while retaining performance on VQA. We suggest this work as a move towards robustness by embedding logical connectives in visual understanding.