# An Analysis of a BERT Deep Learning Strategy on a Technology Assisted Review Task

arXiv: https://arxiv.org/abs/2104.08340 · HTML: https://mlsanity.com/paper/2104.08340/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 36 references that are in the map.
- First posted on arXiv 16 Apr 2021.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks](https://mlsanity.com/paper/2010.08240/) (2020)
- [BERT for Evidence Retrieval and Claim Verification](https://mlsanity.com/paper/1910.02655/) (2019)
- [Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval](https://mlsanity.com/paper/1502.06922/) (2015)
- [VGCN-BERT: Augmenting BERT with Graph Embedding for Text Classification](https://mlsanity.com/paper/2004.05707/) (2020)
- [Simple Applications of BERT for Ad Hoc Document Retrieval](https://mlsanity.com/paper/1903.10972/) (2019)

## Built on by (most cited first, 1 of 1)

- [Goldilocks: Just-Right Tuning of BERT for Technology-Assisted Review](https://mlsanity.com/paper/2105.01044/) (2021)

## Abstract (arXiv)

Document screening is a central task within Evidenced Based Medicine, which is a clinical discipline that supplements scientific proof to back medical decisions. Given the recent advances in DL (Deep Learning) methods applied to Information Retrieval tasks, I propose a DL document classification approach with BERT or PubMedBERT embeddings and a DL similarity search path using SBERT embeddings to reduce physicians' tasks of screening and classifying immense amounts of documents to answer clinical queries. I test and evaluate the retrieval effectiveness of my DL strategy on the 2017 and 2018 CLEF eHealth collections. I find that the proposed DL strategy works, I compare it to the recently successful BM25 plus RM3 model, and conclude that the suggested method accomplishes advanced retrieval performance in the initial ranking of the articles with the aforementioned datasets, for the CLEF eHealth Technologically Assisted Reviews in Empirical Medicine Task.