# Learning by Asking Questions

arXiv: https://arxiv.org/abs/1712.01238 · HTML: https://mlsanity.com/paper/1712.01238/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2017.

## Built on by (most cited first, 25 of 25)

- [Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering](https://mlsanity.com/paper/2107.02331/) (2021)
- [Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language](https://mlsanity.com/paper/2110.15358/) (2021)
- [Learning to Execute Actions or Ask Clarification Questions](https://mlsanity.com/paper/2204.08373/) (2022)
- [LOVA3: Learning to Visual Question Answering, Asking and Assessment](https://mlsanity.com/paper/2405.14974/) (2024)
- [Open-domain clarification question generation without question examples](https://mlsanity.com/paper/2110.09779/) (2021)
- [Learning to Retrieve Videos by Asking Questions](https://mlsanity.com/paper/2205.05739/) (2022)
- [Right this way: Can VLMs Guide Us to See More to Answer Questions?](https://mlsanity.com/paper/2411.00394/) (2024)
- [K-VQG: Knowledge-aware Visual Question Generation for Common-sense Acquisition](https://mlsanity.com/paper/2203.07890/) (2022)
- [Single-Modal Entropy based Active Learning for Visual Question Answering](https://mlsanity.com/paper/2110.10906/) (2021)
- [From Pixels to Graphs: using Scene and Knowledge Graphs for HD-EPIC VQA Challenge](https://mlsanity.com/paper/2506.08553/) (2025)
- [KANDINSKYPatterns -- An experimental exploration environment for Pattern Analysis and Machine Intelligence](https://mlsanity.com/paper/2103.00519/) (2021)
- [Look, Read and Ask: Learning to Ask Questions by Reading Text in Images](https://mlsanity.com/paper/2211.12950/) (2022)
- [Field-Guide-Inspired Zero-Shot Learning](https://mlsanity.com/paper/2108.10967/) (2021)
- [From Wrong To Right: A Recursive Approach Towards Vision-Language Explanation](https://mlsanity.com/paper/2311.12391/) (2023)
- [Learning to Explore Informative Trajectories and Samples for Embodied Perception](https://mlsanity.com/paper/2303.10936/) (2023)
- [Learning What Matters: Prioritized Concept Learning via Relative Error-driven Sample Selection](https://mlsanity.com/paper/2506.01085/) (2025)
- [Learning by Asking Questions for Knowledge-based Novel Object Recognition](https://mlsanity.com/paper/2210.05879/) (2022)
- [Advancing Large Multi-modal Models with Explicit Chain-of-Reasoning and Visual Question Generation](https://mlsanity.com/paper/2401.10005/) (2024)
- [Diagram-Driven Course Questions Generation](https://mlsanity.com/paper/2411.17771/) (2024)
- [Long-context Non-factoid Question Answering in Indic Languages](https://mlsanity.com/paper/2504.13615/) (2025)
- [Ask Before You Act: Generalising to Novel Environments by Asking Questions](https://mlsanity.com/paper/2209.04665/) (2022)
- [CAUS: A Dataset for Question Generation based on Human Cognition Leveraging Large Language Models](https://mlsanity.com/paper/2404.11835/) (2024)
- [Meaning Versus Information, Prediction Versus Memory, and Question Versus Answer](https://mlsanity.com/paper/2107.13393/) (2021)
- [Weakly Supervised Visual Question Answer Generation](https://mlsanity.com/paper/2306.06622/) (2023)
- [Active Learning Methods for Efficient Data Utilization and Model Performance Enhancement](https://mlsanity.com/paper/2504.16136/) (2025)

## Abstract (arXiv)

We introduce an interactive learning framework for the development and testing of intelligent visual systems, called learning-by-asking (LBA). We explore LBA in context of the Visual Question Answering (VQA) task. LBA differs from standard VQA training in that most questions are not observed during training time, and the learner must ask questions it wants answers to. Thus, LBA more closely mimics natural learning and has the potential to be more data-efficient than the traditional VQA setting. We present a model that performs LBA on the CLEVR dataset, and show that it automatically discovers an easy-to-hard curriculum when learning interactively from an oracle. Our LBA generated data consistently matches or outperforms the CLEVR train data and is more sample efficient. We also show that our model asks questions that generalize to state-of-the-art VQA models and to novel test time distributions.