# Neural Rankers for Effective Screening Prioritisation in Medical Systematic Review Literature Search

arXiv: https://arxiv.org/abs/2212.09017 · HTML: https://mlsanity.com/paper/2212.09017/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 58 references that are in the map.
- First posted on arXiv 18 Dec 2022.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets](https://mlsanity.com/paper/1906.05474/) (2019)
- [Multi-Stage Document Ranking with BERT](https://mlsanity.com/paper/1910.14424/) (2019)
- [Condenser: a Pre-training Architecture for Dense Retrieval](https://mlsanity.com/paper/2104.08253/) (2021)
- [BERT with History Answer Embedding for Conversational Question Answering](https://mlsanity.com/paper/1905.05412/) (2019)
- [Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipeline](https://mlsanity.com/paper/2101.08751/) (2021)
- [Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback](https://mlsanity.com/paper/2108.13454/) (2021)
- [Pseudo-Relevance Feedback for Multiple Representation Dense Retrieval](https://mlsanity.com/paper/2106.11251/) (2021)
- [Pseudo Relevance Feedback with Deep Language Models and Dense Retrievers: Successes and Pitfalls](https://mlsanity.com/paper/2108.11044/) (2021)
- [Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback: A Reproducibility Study](https://mlsanity.com/paper/2112.06400/) (2021)
- [Goldilocks: Just-Right Tuning of BERT for Technology-Assisted Review](https://mlsanity.com/paper/2105.01044/) (2021)
- [To Interpolate or not to Interpolate: PRF, Dense and Sparse Retrievers](https://mlsanity.com/paper/2205.00235/) (2022)
- [From Little Things Big Things Grow: A Collection with Seed Studies for Medical Systematic Review Literature Search](https://mlsanity.com/paper/2204.03096/) (2022)
- [Seed-driven Document Ranking for Systematic Reviews: A Reproducibility Study](https://mlsanity.com/paper/2112.04090/) (2021)
- [Search Strategy Formulation for Systematic Reviews: issues, challenges and opportunities](https://mlsanity.com/paper/2112.09424/) (2021)
- [Automated MeSH Term Suggestion for Effective Query Formulation in Systematic Reviews Literature Search](https://mlsanity.com/paper/2209.08687/) (2022)
- [How does Feedback Signal Quality Impact Effectiveness of Pseudo Relevance Feedback for Passage Retrieval?](https://mlsanity.com/paper/2205.05888/) (2022)

## Built on by (most cited first, 9 of 9)

- [The emergence of Large Language Models (LLM) as a tool in literature reviews: an LLM automated systematic review](https://mlsanity.com/paper/2409.04600/) (2024)
- [Zero-shot Generative Large Language Models for Systematic Review Screening Automation](https://mlsanity.com/paper/2401.06320/) (2024)
- [Bio-SIEVE: Exploring Instruction Tuning Large Language Models for Systematic Review Automation](https://mlsanity.com/paper/2308.06610/) (2023)
- [Generating Natural Language Queries for More Effective Systematic Review Screening Prioritisation](https://mlsanity.com/paper/2309.05238/) (2023)
- [2D Matryoshka Training for Information Retrieval](https://mlsanity.com/paper/2411.17299/) (2024)
- [Pre-training vs. Fine-tuning: A Reproducibility Study on Dense Retrieval Knowledge Acquisition](https://mlsanity.com/paper/2505.07166/) (2025)
- [Annotating Data for Fine-Tuning a Neural Ranker? Current Active Learning Strategies are not Better than Random Selection](https://mlsanity.com/paper/2309.06131/) (2023)
- [Document Set Expansion with Positive-Unlabeled Learning: A Density Estimation-based Approach](https://mlsanity.com/paper/2401.11145/) (2024)
- [Harnessing PubMed User Query Logs for Post Hoc Explanations of Recommended Similar Articles](https://mlsanity.com/paper/2402.03484/) (2024)

## Abstract (arXiv)

Medical systematic reviews typically require assessing all the documents retrieved by a search. The reason is two-fold: the task aims for ``total recall''; and documents retrieved using Boolean search are an unordered set, and thus it is unclear how an assessor could examine only a subset. Screening prioritisation is the process of ranking the (unordered) set of retrieved documents, allowing assessors to begin the downstream processes of the systematic review creation earlier, leading to earlier completion of the review, or even avoiding screening documents ranked least relevant. Screening prioritisation requires highly effective ranking methods. Pre-trained language models are state-of-the-art on many IR tasks but have yet to be applied to systematic review screening prioritisation. In this paper, we apply several pre-trained language models to the systematic review document ranking task, both directly and fine-tuned. An empirical analysis compares how effective neural methods compare to traditional methods for this task. We also investigate different types of document representations for neural methods and their impact on ranking performance. Our results show that BERT-based rankers outperform the current state-of-the-art screening prioritisation methods. However, BERT rankers and existing methods can actually be complementary, and thus, further improvements may be achieved if used in conjunction.