# Multimodal QUD: Inquisitive Questions from Scientific Figures

arXiv: https://arxiv.org/abs/2604.23733 · HTML: https://mlsanity.com/paper/2604.23733/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 60 of its 99 references that are in the map.
- First posted on arXiv 26 Apr 2026.

## Builds on (most cited first, 30 of 60)

- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [PlotQA: Reasoning over Scientific Plots](https://mlsanity.com/paper/1909.00997/) (2019)
- [FigureQA: An Annotated Figure Dataset for Visual Reasoning](https://mlsanity.com/paper/1710.07300/) (2017)
- [SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models](https://mlsanity.com/paper/2307.10635/) (2023)
- [Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models](https://mlsanity.com/paper/2403.00231/) (2024)
- [SAEs Are Good for Steering -- If You Select the Right Features](https://mlsanity.com/paper/2505.20063/) (2025)
- [SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers](https://mlsanity.com/paper/2407.09413/) (2024)
- [SciGraphQA: A Large-Scale Synthetic Multi-Turn Question-Answering Dataset for Scientific Graphs](https://mlsanity.com/paper/2308.03349/) (2023)
- [IsoScore: Measuring the Uniformity of Embedding Space Utilization](https://mlsanity.com/paper/2108.07344/) (2021)
- [SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretation](https://mlsanity.com/paper/2405.08807/) (2024)
- [Controllable Open-ended Question Generation with A New Question Type Ontology](https://mlsanity.com/paper/2107.00152/) (2021)
- [Inquisitive Question Generation for High Level Text Comprehension](https://mlsanity.com/paper/2010.01657/) (2020)
- [ReFACT: Updating Text-to-Image Models by Editing the Text Encoder](https://mlsanity.com/paper/2306.00738/) (2023)
- [ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models](https://mlsanity.com/paper/2505.13444/) (2025)
- [Pixels Versus Priors: Controlling Knowledge Priors in Vision-Language Models through Visual Counterfacts](https://mlsanity.com/paper/2505.17127/) (2025)
- [What Do VLMs NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Corruption and Evaluation](https://mlsanity.com/paper/2406.16320/) (2024)
- [MedBrowseComp: Benchmarking Medical Deep Research and Computer Use](https://mlsanity.com/paper/2505.14963/) (2025)
- [The Best of Both Worlds: Accurate Global and Personalized Models through Federated Learning with Data-Free Hyper-Knowledge Distillation](https://mlsanity.com/paper/2301.08968/) (2023)
- [Enhancing Retrieval-Augmented Generation: A Study of Best Practices](https://mlsanity.com/paper/2501.07391/) (2025)
- [Can Language Models Learn Typologically Implausible Languages?](https://mlsanity.com/paper/2502.12317/) (2025)
- [PeerQA: A Scientific Question Answering Dataset from Peer Reviews](https://mlsanity.com/paper/2502.13668/) (2025)
- [Discourse Comprehension: A Question Answering Framework to Represent Sentence Connections](https://mlsanity.com/paper/2111.00701/) (2021)
- [QUDEVAL: The Evaluation of Questions Under Discussion Discourse Parsing](https://mlsanity.com/paper/2310.14520/) (2023)
- [SpaceGNN: Multi-Space Graph Neural Network for Node Anomaly Detection with Extremely Limited Labels](https://mlsanity.com/paper/2502.03201/) (2025)
- [Forgotten Polygons: Multimodal Large Language Models are Shape-Blind](https://mlsanity.com/paper/2502.15969/) (2025)
- [Viziometrics: Analyzing Visual Information in the Scientific Literature](https://mlsanity.com/paper/1605.04951/) (2016)
- [COMPS: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models](https://mlsanity.com/paper/2210.01963/) (2022)
- [Cross-Care: Assessing the Healthcare Implications of Pre-training Data on Language Model Bias](https://mlsanity.com/paper/2405.05506/) (2024)
- [Elaborative Simplification as Implicit Questions Under Discussion](https://mlsanity.com/paper/2305.10387/) (2023)

## Abstract (arXiv)

Discourse comprehension in complex documents often involves continuously posing and resolving Questions Under Discussion (QUDs). While QUD frameworks have so far focused on text, scientific literature is inherently multimodal: figures convey discourse goals distinct from their textual counterparts, thus invoking implicit questions that the surrounding text answers. In scientific discovery, knowing the right questions to ask is as important as knowing how to answer them, yet this capability remains largely absent in current models. In this work, we extend QUD to multimodal discourse in scientific literature, targeting questions evoked by figures that are (1) inquisitive, i.e., not resolved in the prior context; (2) salient, i.e., relevant to the paper's research claims and addressed later in the paper; (3) grounded in visual insights. To benchmark model capability to generate such questions, we introduce MQUD, a dataset of 1,250 figure-evoked questions from 56 scientific papers, including 708 questions annotated by the original authors of these papers. Our experiments show that open-source VLMs such as Qwen 3.5 predominantly ask questions that can be answered by the figure alone. Fine-tuning on MQUD teaches the model to ask scientifically relevant, inquisitive questions that target the role a figure plays in the paper's argument.