mlsanityData to 7 Oct 2026

Clinfo.ai: An Open-Source Retrieval-Augmented Large Language Model System for Answering Medical Questions using Scientific Literature

arXiv 2310.16146 ↗First posted 24 Oct 2023cs.IR cs.AI cs.CLcited by 29

On the map: Retrieval-augmented generation → Medical and biomedical RAG

Abstract (arXiv)

The quickly-expanding nature of published medical literature makes it challenging for clinicians and researchers to keep up with and summarize recent, relevant findings in a timely manner. While several closed-source summarization tools based on large language models (LLMs) now exist, rigorous and systematic evaluations of their outputs are lacking. Furthermore, there is a paucity of high-quality datasets and appropriate benchmark tasks with which to evaluate these tools. We address these issues with four contributions: we release Clinfo.ai, an open-source WebApp that answers clinical questions based on dynamically retrieved scientific literature; we specify an information retrieval and abstractive summarization task to evaluate the performance of such retrieval-augmented LLM systems; we release a dataset of 200 questions and corresponding answers derived from published systematic reviews, which we name PubMed Retrieval and Synthesis (PubMedRS-200); and report benchmark results for Clinfo.ai and other publicly available OpenQA systems on PubMedRS-200.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in Medical and biomedical RAG, one of 15 directions in Retrieval-augmented generation. Builds on 19 of its 59 references that are in the map. First posted on arXiv 24 Oct 2023.

Builds on

19 of 59 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Improving accuracy of GPT-3/4 results on biomedical data using a retrieval-augmented language model26 May

Other references · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  7. 2023A Survey of Large Language Modelscited by 2,537
  8. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  9. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  10. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  11. 2022Memory-Based Model Editing at Scalecited by 443
  12. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
6 more
  1. 2022Towards a Unified Multi-Dimensional Evaluator for Text Generationcited by 264
  2. 2022Fine-tuning language models to find agreement among humans with diverse preferencescited by 189
  3. 2021Compression, Transduction, and Creation: A Unified Framework for Evaluating Natural Language Generationcited by 60
  4. 2023MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Recordscited by 54
  5. 2020Open Source Software for Efficient and Transparent Reviewscited by 34
  6. 2023NLG Evaluation Metrics Beyond Correlation Analysis: An Empirical Metric Preference Checklistcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 14; 2026: 4.

29 papers.

Sort:

By area:

  1. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
  2. 2024Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questionscited by 54
  3. 2024DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer's Disease Questions with Scientific Literaturecited by 50
  4. 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
  5. 2025BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and Vision-Language Models Derived from Scientific Literaturecited by 34
  6. 2025LettuceDetect: A Hallucination Detection Framework for RAG Applicationscited by 32
  7. 2024The emergence of Large Language Models (LLM) as a tool in literature reviews: an LLM automated systematic reviewcited by 20
  8. 2025Silent Leaks: Implicit Knowledge Extraction Attack on RAG Systems through Benign Queriescited by 19
  9. 2024PersonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideationcited by 16
  10. 2024ERAGent: Enhancing Retrieval-Augmented Language Models with Improved Accuracy, Efficiency, and Personalizationcited by 15
  11. 2025MedCite: Can Language Models Generate Verifiable Text for Medicine?cited by 11
  12. 2025Defending Against Knowledge Poisoning Attacks During Retrieval-Augmented Generationcited by 11
17 more
  1. 2025CG-RAG: Research Question Answering by Citation Graph Retrieval-Augmented LLMscited by 8
  2. 2025Bring Your Own Knowledge: A Survey of Methods for LLM Knowledge Expansioncited by 8
  3. 2024MedInsight: A Multi-Source Context Augmentation Framework for Generating Patient-Centric Medical Responses using Large Language Modelscited by 7
  4. 2025Can Large Language Models Match the Conclusions of Systematic Reviews?cited by 7
  5. 2024WeKnow-RAG: An Adaptive Approach for Retrieval-Augmented Generation Integrating Web Search and Knowledge Graphscited by 6
  6. 2025From Promising Capability to Pervasive Bias: Assessing Large Language Models for Emergency Department Triagecited by 5
  7. 2025BioMol-MQA: A Multi-Modal Question Answering Dataset For LLM Reasoning Over Bio-Molecular Interactionscited by 4
  8. 2025REIS: A High-Performance and Energy-Efficient Retrieval System with In-Storage Processingcited by 4
  9. 2026TARSE: Test-Time Adaptation via Retrieval of Skills and Experience for Reasoning Agentscited by 4
  10. 2025EICopilot: Search and Explore Enterprise Information over Large-scale Knowledge Graphs with LLM-driven Agentscited by 2
  11. 2025A Large-Scale Vision-Language Dataset Derived from Open Scientific Literature to Advance Biomedical Generalist AIcited by 2
  12. 2025LLMs for Supply Chain Managementcited by 2
  13. 2024Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMscited by 1
  14. 2026ACL-Verbatim: hallucination-free question answering for researchcited by 1
  15. 2026Inference Cost Attacks for Retrieval-Augmented Large Language Modelscited by 1
  16. 2024Building Efficient and Effective OpenQA Systems for Low-Resource Languagescited by 0
  17. 2026Ten Headache Specialists versus Artificial Intelligence for Clinical Literature Summarization: A Critical Evaluation and Comparisoncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Evaluating Large Language Models for Evidence-Based Clinical Question Answering
  2. 2026A corpus-specific clinical RAG system matches or outperforms newer frontier LLMs on HealthBench
  3. 2025Optimizing Medical Question-Answering Systems: A Comparative Study of Fine-Tuned and Zero-Shot Large Language Models with RAG Framework
  4. 2025IMB: An Italian Medical Benchmark for Question Answering
  5. 2024A Review on Scientific Knowledge Extraction using Large Language Models in Biomedical Sciences
  6. 2025Retrieval-Augmented Generation in Medicine: A Scoping Review of Technical Implementations, Clinical Applications, and Ethical Considerations
  7. 2026Harnessing Large Language Models for Precision Querying and Retrieval-Augmented Knowledge Extraction in Clinical Data Science
  8. 2023MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answering