mlsanityData to 7 Oct 2026

Query-driven Document-level Scientific Evidence Extraction from Biomedical Studies

arXiv 2505.06186 ↗First posted 9 May 2025cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Extracting scientific evidence from biomedical studies for clinical research questions (e.g., Does stem cell transplantation improve quality of life in patients with medically refractory Crohn's disease compared to placebo?) is a crucial step in synthesising biomedical evidence. In this paper, we focus on the task of document-level scientific evidence extraction for clinical questions with conflicting evidence. To support this task, we create a dataset called CochraneForest, leveraging forest plots from Cochrane systematic reviews. It comprises 202 annotated forest plots, associated clinical research questions, full texts of studies, and study-specific conclusions. Building on CochraneForest, we propose URCA (Uniform Retrieval Clustered Augmentation), a retrieval-augmented generation framework designed to tackle the unique challenges of evidence extraction. Our experiments show that URCA outperforms the best existing methods by up to 10.3% in F1 score on this task. However, the results also underscore the complexity of CochraneForest, establishing it as a challenging testbed for advancing automated evidence synthesis systems.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 27 of its 49 references that are in the map. First posted on arXiv 9 May 2025.

Builds on

27 of 49 references.

References · 27

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  8. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  9. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  10. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  11. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  12. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
15 more
  1. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
  2. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
  3. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  4. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
  5. 2022RealTime QA: What's the Answer Right Now?cited by 239
  6. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 144
  7. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  8. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 97
  9. 2019Inferring Which Medical Treatments Work from Reports of Clinical Trialscited by 63
  10. 2024Scaling Retrieval-Based Language Models with a Trillion-Token Datastorecited by 54
  11. 2023Summarizing, Simplifying, and Synthesizing Medical Evidence Using GPT-3 (with Varying Success)cited by 40
  12. 2023Can large language models replace humans in the systematic review process? Evaluating GPT-4's efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languagescited by 36
  13. 2023Appraising the Potential Uses and Harms of LLMs for Medical Systematic Reviewscited by 18
  14. 2020Trialstreamer: Mapping and Browsing Medical Evidence in Real-Timecited by 12
  15. 2022Detecting Contradictory COVID-19 Drug Efficacy Claims from Biomedical Literaturecited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025TaxoAlign: Scholarly Taxonomy Generation Using Language Modelscited by 2
  2. 2026AutoForest: Automatically Generating Forest Plots from Biomedical Studies with End-to-End Evidence Extraction and Synthesiscited by 1
  3. 2025SciEvent: Benchmarking Multi-domain Scientific Event Extractioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Review on Scientific Knowledge Extraction using Large Language Models in Biomedical Sciences
  2. 2026A Hybrid Retrieval and Reranking Framework for Evidence-Grounded Retrieval-Augmented Generation
  3. 2026Building evidence-based knowledge bases from full-text literature for disease-specific biomedical reasoning
  4. 2026Adaptive Retrieval Strategies for Biomedical Question Answering
  5. 2025Introducing Answered with Evidence -- a framework for evaluating whether LLM responses to biomedical questions are founded in evidence
  6. 2025Rethinking Retrieval-Augmented Generation for Medicine: A Large-Scale, Systematic Expert Evaluation and Practical Insights
  7. 2026When Retrieval Helps and Distracts: Evaluating Evidence-Generating LLMs for Biomedical Claim Verification
  8. 2026Evaluating Biomedical Reranking for LLM-Based Question Answering over Longitudinal Clinical Notes