mlsanityData to 14 Oct 2026

BioMol-MQA: A Multi-Modal Question Answering Dataset For LLM Reasoning Over Bio-Molecular Interactions

arXiv 2506.05766 ↗First posted 6 Jun 2025cs.CLcited by 4

On the map: LLM reasoning → Clinical and biomedical applications

also in 1 more
Abstract (arXiv)

Retrieval augmented generation (RAG) has shown great power in improving Large Language Models (LLMs). However, most existing RAG-based LLMs are dedicated to retrieving single modality information, mainly text; while for many real-world problems, such as healthcare, information relevant to queries can manifest in various modalities such as knowledge graph, text (clinical notes), and complex molecular structure. Thus, being able to retrieve relevant multi-modality domain-specific information, and reason and synthesize diverse knowledge to generate an accurate response is important. To address the gap, we present BioMol-MQA, a new question-answering (QA) dataset on polypharmacy, which is composed of two parts (i) a multimodal knowledge graph (KG) with text and molecular structure for information retrieval; and (ii) challenging questions that designed to test LLM capabilities in retrieving and reasoning over multimodal KG to answer questions. Our benchmarks indicate that existing LLMs struggle to answer these questions and do well only when given the necessary background data, signaling the necessity for strong RAG frameworks.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 92 of its 142 references that are in the map. First posted on arXiv 6 Jun 2025.

Builds on

92 of 142 references.

References · 92

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2017Graph Attention Networkscited by 8,923
  6. 2025Qwen3 Technical Reportcited by 8,820
  7. 2017Inductive Representation Learning on Large Graphscited by 6,320
  8. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  9. 2024GPT-4o System Cardcited by 4,588
  10. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  11. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  12. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
38 more
  1. 2023Mistral 7Bcited by 3,164
  2. 2023A Survey of Large Language Modelscited by 2,576
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  4. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  5. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  6. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
  7. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  8. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
  9. 2024A Survey on LLM-as-a-Judgecited by 1,423
  10. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  11. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  12. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  13. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  14. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  15. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  16. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  17. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  18. 2023Active Retrieval Augmented Generationcited by 516
  19. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  20. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
  21. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
  22. 2023Benchmarking Large Language Models for News Summarizationcited by 445
  23. 2018Modeling polypharmacy side effects with graph convolutional networkscited by 237
  24. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 223
  25. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Textcited by 220
  26. 2021QuALITY: Question Answering with Long Input Texts, Yes!cited by 208
  27. 2023What can Large Language Models do in chemistry? A comprehensive benchmark on eight taskscited by 185
  28. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 185
  29. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 184
  30. 2021MultiModalQA: Complex Question Answering over Text, Tables and Imagescited by 179
  31. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 166
  32. 2023Benchmarking Cognitive Biases in Large Language Models as Evaluatorscited by 150
  33. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  34. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 138
  35. 2024GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoningcited by 128
  36. 2022ReACC: A Retrieval-Augmented Code Completion Frameworkcited by 119
  37. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systemscited by 117
  38. 2024CRAG -- Comprehensive RAG Benchmarkcited by 97

50 most cited of 92 All 92 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Generalizing Test-time Compute-optimal Scaling as an Optimizable Graphcited by 4
  2. 2026Query-Efficient Agentic Graph Extraction Attacks on GraphRAG Systemscited by 3
  3. 2026URAG: A Benchmark for Uncertainty Quantification in Retrieval-Augmented Large Language Modelscited by 3
  4. 2026Multi-Modal Generative Fuzzy System: Fuzzy Inference Guided Large Model Interactive Question Answering Frameworkcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Harnessing Collective Intelligence of LLMs for Robust Biomedical QA: A Multi-Model Approach
  2. 2025MedBioLM: Optimizing Medical and Biological QA with Fine-Tuned Large Language Models and Retrieval-Augmented Generation
  3. 2025BioMedSearch: A Multi-Source Biomedical Retrieval Framework Based on LLMs
  4. 2026From Voting to Agent Collaboration: Answer-Type-Aware LLM Pipelines for BioASQ 14b
  5. 2024BioRAG: A RAG-LLM Framework for Biological Question Reasoning
  6. 2023MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answering
  7. 2025RAG-BioQA: A Retrieval-Augmented Generation Framework for Long-Form Biomedical Question Answering
  8. 2024Tool Calling: Enhancing Medication Consultation via Retrieval-Augmented Large Language Models