# Querying Multimodal Scientific Papers with AI: Practices and Preferences Across Blind, Low-Vision, and Sighted Scientists

arXiv: https://arxiv.org/abs/2607.18514 · HTML: https://mlsanity.com/paper/2607.18514/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 55 references that are in the map.
- First posted on arXiv 20 Jul 2026.

## Builds on (most cited first, 15 of 15)

- [MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?](https://mlsanity.com/paper/2403.14624/) (2024)
- [SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers](https://mlsanity.com/paper/2407.09413/) (2024)
- [Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content](https://mlsanity.com/paper/2110.04406/) (2021)
- [M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Models](https://mlsanity.com/paper/2411.04075/) (2024)
- [Viziometrics: Analyzing Visual Information in the Scientific Literature](https://mlsanity.com/paper/1605.04951/) (2016)
- [VizAbility: Enhancing Chart Accessibility with LLM-based Conversational Interaction](https://mlsanity.com/paper/2310.09611/) (2023)
- [A Dataset of Alt Texts from HCI Publications: Analyses and Uses Towards Producing More Descriptive Alt Texts of Data Visualizations in Scientific Papers](https://mlsanity.com/paper/2209.13718/) (2022)
- [Can Multimodal Foundation Models Understand Schematic Diagrams? An Empirical Study on Information-Seeking QA over Scientific Papers](https://mlsanity.com/paper/2507.10787/) (2025)
- [Improving the Accessibility of Scientific Documents: Current State, User Needs, and a System Solution to Enhance Scientific PDF Accessibility for Blind and Low Vision Users](https://mlsanity.com/paper/2105.00076/) (2021)
- [Uncovering the New Accessibility Crisis in Scholarly PDFs](https://mlsanity.com/paper/2410.03022/) (2024)
- [Engaging with Children's Artwork in Mixed Visual-Ability Families](https://mlsanity.com/paper/2407.18874/) (2024)
- [Sightation Counts: Leveraging Sighted User Feedback in Building a BLV-aligned Dataset of Diagram Descriptions](https://mlsanity.com/paper/2503.13369/) (2025)
- [How Multimodal Large Language Models Support Access to Visual Information: A Diary Study With Blind and Low Vision People](https://mlsanity.com/paper/2602.13469/) (2026)
- ["Accessibility people, you go work on that thing of yours over there": Addressing Disability Inclusion in AI Product Organizations](https://mlsanity.com/paper/2508.16607/) (2025)
- [GeoVisA11y: An AI-based Geovisualization Question-Answering System for Screen-Reader Users](https://mlsanity.com/paper/2603.07446/) (2026)

## Built on by (most cited first, 1 of 1)

- [greCAPTCHA: Assessing Understanding as Evidence of Research Authorship Under Generative AI](https://mlsanity.com/paper/2609.20481/) (2026)

## Abstract (arXiv)

Visual diagrams, figures, and tables are central to scientific papers, and convey information beyond what is captured in text. While blind or low-vision (BLV) scientists have traditionally relied on static alternative text to access figures in papers, the rise of artificial intelligence (AI) has made interactive question-answering (QA) a feasible paradigm for visual exploration; yet little is known about how scientists use visual QA in practice or how to improve its accessibility. In this work, we interview five BLV and five sighted scientists across different STEM fields to understand how they use two AI tools, ChatGPT and Gemini, to query multimodal scientific documents. Our findings characterize how scientists review multimodal content, including existing practices (along with accessibility workarounds) for engaging with visuals, and feedback on the suitability of AI-generated responses to multimodal queries. We further find that vague or incomplete image descriptions, as well as incorrect AI outputs more broadly, can cause both BLV and sighted scientists to abandon AI workflows. To support future research, we additionally contribute a dataset of 115 queries and responses from our participants' interactions with the AI tools for papers in their field. We close by discussing implications for AI-powered scientific QA systems, emphasizing considerations for access across abilities and domains.