# Actively Seeking and Learning from Live Data

arXiv: https://arxiv.org/abs/1904.02865 · HTML: https://mlsanity.com/paper/1904.02865/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Apr 2019.

## Built on by (most cited first, 9 of 9)

- [Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answering](https://mlsanity.com/paper/2104.03149/) (2021)
- [Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledge](https://mlsanity.com/paper/2101.06013/) (2021)
- [Robust Visual Question Answering: Datasets, Methods, and Future Challenges](https://mlsanity.com/paper/2307.11471/) (2023)
- [Visual Perturbation-aware Collaborative Learning for Overcoming the Language Prior Problem](https://mlsanity.com/paper/2207.11850/) (2022)
- [Discovering the Unknown Knowns: Turning Implicit Knowledge in the Dataset into Explicit Training Examples for Visual Question Answering](https://mlsanity.com/paper/2109.06122/) (2021)
- [Visual Question Answering: which investigated applications?](https://mlsanity.com/paper/2103.02937/) (2021)
- [X-GGM: Graph Generative Modeling for Out-of-Distribution Generalization in Visual Question Answering](https://mlsanity.com/paper/2107.11576/) (2021)
- [Language bias in Visual Question Answering: A Survey and Taxonomy](https://mlsanity.com/paper/2111.08531/) (2021)
- [Zero-shot Retrieval: Augmenting Pre-trained Models with Search Engines](https://mlsanity.com/paper/2311.17949/) (2023)

## Abstract (arXiv)

One of the key limitations of traditional machine learning methods is their requirement for training data that exemplifies all the information to be learned. This is a particular problem for visual question answering methods, which may be asked questions about virtually anything. The approach we propose is a step toward overcoming this limitation by searching for the information required at test time. The resulting method dynamically utilizes data from an external source, such as a large set of questions/answers or images/captions. Concretely, we learn a set of base weights for a simple VQA model, that are specifically adapted to a given question with the information specifically retrieved for this question. The adaptation process leverages recent advances in gradient-based meta learning and contributions for efficient retrieval and cross-domain adaptation. We surpass the state-of-the-art on the VQA-CP v2 benchmark and demonstrate our approach to be intrinsically more robust to out-of-distribution test data. We demonstrate the use of external non-VQA data using the MS COCO captioning dataset to support the answering process. This approach opens a new avenue for open-domain VQA systems that interface with diverse sources of data.