# SIQA: Toward Reliable Scientific Image Quality Assessment arXiv: https://arxiv.org/abs/2603.06700 ยท HTML: https://mlsanity.com/paper/2603.06700/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Builds on 27 of its 43 references that are in the map. - First posted on arXiv 5 Mar 2026. ## Builds on (most cited first, 27 of 27) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023) - [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024) - [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023) - [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025) - [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025) - [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022) - [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025) - [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017) - [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022) - [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023) - [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024) - [Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels](https://mlsanity.com/paper/2312.17090/) (2023) - [Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment](https://mlsanity.com/paper/1612.01697/) (2016) - [Massive Online Crowdsourced Study of Subjective and Objective Picture Quality](https://mlsanity.com/paper/1511.02919/) (2015) - [Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models](https://mlsanity.com/paper/2311.06783/) (2023) - [PaperQA: Retrieval-Augmented Generative Agent for Scientific Research](https://mlsanity.com/paper/2312.07559/) (2023) - [AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment](https://mlsanity.com/paper/2306.04717/) (2023) - [Learning Multi-dimensional Human Preference for Text-to-Image Generation](https://mlsanity.com/paper/2405.14705/) (2024) - [KOSMOS-2.5: A Multimodal Literate Model](https://mlsanity.com/paper/2309.11419/) (2023) - [InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery](https://mlsanity.com/paper/2311.16208/) (2023) - [ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area](https://mlsanity.com/paper/2408.07246/) (2024) - [GenExam: A Multidisciplinary Text-to-Image Exam](https://mlsanity.com/paper/2509.14232/) (2025) - [Subjective and Objective Quality Assessment for in-the-Wild Computer Graphics Images](https://mlsanity.com/paper/2303.08050/) (2023) - [BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset](https://mlsanity.com/paper/2507.03483/) (2025) - [PEACE: Empowering Geologic Map Holistic Understanding with MLLMs](https://mlsanity.com/paper/2501.06184/) (2025) - [Q-Mirror: Unlocking the Multi-Modal Potential of Scientific Text-Only QA Pairs](https://mlsanity.com/paper/2509.24297/) (2025) ## Built on by (most cited first, 4 of 4) - [Towards Characterizing Scientific Image Utility and Upgradability](https://mlsanity.com/paper/2606.03401/) (2026) - [SciFigQual-Bench: A Benchmark for Scientific Figure Quality Assessment with Full-Manuscript Context](https://mlsanity.com/paper/2607.27084/) (2026) - [SciQNet: Two-Stage Multimodal Adaptation for Scientific Image Quality Assessment](https://mlsanity.com/paper/2608.05691/) (2026) - [Scientific Image Quality Assessment via Multi-modal Retrieval-Augmented Generation](https://mlsanity.com/paper/2609.19634/) (2026) ## Abstract (arXiv) Scientific images fundamentally differ from natural and AI-generated images in that they encode structured domain knowledge rather than merely depict visual scenes. Assessing their quality therefore requires evaluating not only perceptual fidelity but also scientific correctness and logical completeness. However, existing image quality assessment (IQA) paradigms primarily focus on perceptual distortions or image-text alignment, implicitly assuming that depicted content is factually valid. This assumption breaks down in scientific contexts, where visually plausible figures may still contain conceptual errors or incomplete reasoning. To address this gap, we introduce Scientific Image Quality Assessment (SIQA), a framework that models scientific image quality along two complementary dimensions: Knowledge (Scientific Validity and Scientific Completeness) and Perception (Cognitive Clarity and Disciplinary Conformity). To operationalize this formulation, we design two evaluation protocols: SIQA-U (Understanding), which measures semantic comprehension of scientific content through multiple-choice tasks, and SIQA-S (Scoring), which evaluates alignment with expert quality judgments. We further construct the SIQA Challenge, consisting of an expert-annotated benchmark and a large-scale training set. Experiments across representative multimodal large language models (MLLMs) reveal a consistent discrepancy between scoring alignment and scientific understanding. While models can achieve strong agreement with expert ratings under SIQA-S, their performance on SIQA-U remains substantially lower. Fine-tuning improves both metrics, yet gains in scoring consistently outpace improvements in understanding. These results suggest that rating consistency alone may not reliably reflect scientific comprehension, underscoring the necessity of multidimensional evaluation for scientific image quality assessment.