mlsanityData to 14 Oct 2026

Bio-SIEVE: Exploring Instruction Tuning Large Language Models for Systematic Review Automation

arXiv 2308.06610 ↗First posted 12 Aug 2023cs.CL cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Medical systematic reviews can be very costly and resource intensive. We explore how Large Language Models (LLMs) can support and be trained to perform literature screening when provided with a detailed set of selection criteria. Specifically, we instruction tune LLaMA and Guanaco models to perform abstract screening for medical systematic reviews. Our best model, Bio-SIEVE, outperforms both ChatGPT and trained traditional approaches, and generalises better across medical domains. However, there remains the challenge of adapting the model to safety-first scenarios. We also explore the impact of multi-task training with Bio-SIEVE-Multi, including tasks such as PICO extraction and exclusion reasoning, but find that it is unable to match single-task Bio-SIEVE's performance. We see Bio-SIEVE as an important step towards specialising LLMs for the biomedical systematic review process and explore its future developmental opportunities. We release our models, code and a list of DOIs to reconstruct our dataset for reproducibility.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 22 of its 47 references that are in the map. First posted on arXiv 12 Aug 2023.

Builds on

22 of 47 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2017Deep reinforcement learning from human preferencescited by 3,912
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  8. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  10. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  11. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  12. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
10 more
  1. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  2. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 671
  3. 2023How is ChatGPT's behavior changing over time?cited by 270
  4. 2020Exploring and Predicting Transferability across NLP Taskscited by 128
  5. 2021Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipelinecited by 65
  6. 2023Can ChatGPT Write a Good Boolean Query for Systematic Review Literature Search?cited by 54
  7. 2023Automated Paper Screening for Clinical Reviews Using Large Language Modelscited by 29
  8. 2023Assessing the Ability of ChatGPT to Screen Articles for Systematic Reviewscited by 10
  9. 2022Neural Rankers for Effective Screening Prioritisation in Medical Systematic Review Literature Searchcited by 9
  10. 2022Continuous Active Learning Using Pretrained Transformerscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 2; 2026: 1.

7 papers.

  1. 2024Artificial Intelligence for Literature Reviews: Opportunities and Challengescited by 30
  2. 2024The emergence of Large Language Models (LLM) as a tool in literature reviews: an LLM automated systematic reviewcited by 21
  3. 2024Zero-shot Generative Large Language Models for Systematic Review Screening Automationcited by 14
  4. 2024The Promise and Challenges of Using LLMs to Accelerate the Screening Process of Systematic Reviewscited by 14
  5. 2025From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systemscited by 8
  6. 2025LLM4SCREENLIT: Recommendations on Assessing the Performance of Large Language Models for Screening Literature in Systematic Reviewscited by 5
  7. 2026Preliminary Guidelines for Using and Evaluating GenAI Tools to Support Systematic Literature Reviewscited by 0