# Question Generation for Evaluating Cross-Dataset Shifts in Multi-modal Grounding

arXiv: https://arxiv.org/abs/2201.09639 · HTML: https://mlsanity.com/paper/2201.09639/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 24 Jan 2022.

## Builds on (most cited first, 16 of 16)

- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [VizWiz Grand Challenge: Answering Visual Questions from Blind People](https://mlsanity.com/paper/1802.08218/) (2018)
- [Visual7W: Grounded Question Answering in Images](https://mlsanity.com/paper/1511.03416/) (2015)
- [Words aren't enough, their order matters: On the Robustness of Grounding Visual Referring Expressions](https://mlsanity.com/paper/2005.01655/) (2020)
- [DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering](https://mlsanity.com/paper/2005.00697/) (2020)
- [Evaluating Models' Local Decision Boundaries via Contrast Sets](https://mlsanity.com/paper/2004.02709/) (2020)
- [Cross-Dataset Adaptation for Visual Question Answering](https://mlsanity.com/paper/1806.03726/) (2018)
- [Natural Language Interaction with Explainable AI Models](https://mlsanity.com/paper/1903.05720/) (2019)
- [X-ToM: Explaining with Theory-of-Mind for Gaining Justified Human Trust](https://mlsanity.com/paper/1909.06907/) (2019)
- [Effective Representation to Capture Collaboration Behaviors between Explainer and User](https://mlsanity.com/paper/2201.03147/) (2022)
- [Discourse Analysis for Evaluating Coherence in Video Paragraph Captions](https://mlsanity.com/paper/2201.06207/) (2022)

## Built on by (most cited first, 3 of 3)

- [Grounded Image Text Matching with Mismatched Relation Reasoning](https://mlsanity.com/paper/2308.01236/) (2023)
- [Attention cannot be an Explanation](https://mlsanity.com/paper/2201.11194/) (2022)
- [On the Importance of Karaka Framework in Multi-modal Grounding](https://mlsanity.com/paper/2204.04347/) (2022)

## Abstract (arXiv)

Visual question answering (VQA) is the multi-modal task of answering natural language questions about an input image. Through cross-dataset adaptation methods, it is possible to transfer knowledge from a source dataset with larger train samples to a target dataset where training set is limited. Suppose a VQA model trained on one dataset train set fails in adapting to another, it is hard to identify the underlying cause of domain mismatch as there could exists a multitude of reasons such as image distribution mismatch and question distribution mismatch. At UCLA, we are working on a VQG module that facilitate in automatically generating OOD shifts that aid in systematically evaluating cross-dataset adaptation capabilities of VQA models.