# Efficient Systematic Reviews: Literature Filtering with Transformers & Transfer Learning

arXiv: https://arxiv.org/abs/2405.20354 · HTML: https://mlsanity.com/paper/2405.20354/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 48 references that are in the map.
- First posted on arXiv 30 May 2024.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets](https://mlsanity.com/paper/1906.05474/) (2019)
- [A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literature](https://mlsanity.com/paper/1806.04185/) (2018)
- [Evidence Inference 2.0: More Data, Better Models](https://mlsanity.com/paper/2005.04177/) (2020)
- [Pairwise Multi-Class Document Classification for Semantic Relations between Wikipedia Articles](https://mlsanity.com/paper/2003.09881/) (2020)
- [Advancing PICO Element Detection in Biomedical Text via Deep Neural Networks](https://mlsanity.com/paper/1810.12780/) (2018)
- [Data Mining in Clinical Trial Text: Transformers for Classification and Question Answering Tasks](https://mlsanity.com/paper/2001.11268/) (2020)

## Built on by (most cited first, 1 of 1)

- [Leveraging LLMs for Semi-Automatic Corpus Filtration in Systematic Literature Reviews](https://mlsanity.com/paper/2510.11409/) (2025)

## Abstract (arXiv)

Identifying critical research within the growing body of academic work is an intrinsic aspect of conducting quality research. Systematic review processes used in evidence-based medicine formalise this as a procedure that must be followed in a research program. However, it comes with an increasing burden in terms of the time required to identify the important articles of research for a given topic. In this work, we develop a method for building a general-purpose filtering system that matches a research question, posed as a natural language description of the required content, against a candidate set of articles obtained via the application of broad search terms. Our results demonstrate that transformer models, pre-trained on biomedical literature, and then fine tuned for the specific task, offer a promising solution to this problem. The model can remove large volumes of irrelevant articles for most research questions. Furthermore, analysis of the specific research questions in our training data suggest natural avenues for further improvement.