# Application of Transformers based methods in Electronic Medical Records: A Systematic Literature Review

arXiv: https://arxiv.org/abs/2304.02768 · HTML: https://mlsanity.com/paper/2304.02768/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 95 references that are in the map.
- First posted on arXiv 5 Apr 2023.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [BEHRT: Transformer for Electronic Health Records](https://mlsanity.com/paper/1907.09538/) (2019)
- [Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans](https://mlsanity.com/paper/2008.06388/) (2020)
- [TAPER: Time-Aware Patient EHR Representation](https://mlsanity.com/paper/1908.03971/) (2019)
- [Estimating Redundancy in Clinical Text](https://mlsanity.com/paper/2105.11832/) (2021)
- [Patient Cohort Retrieval using Transformer Language Models](https://mlsanity.com/paper/2009.05121/) (2020)
- [Quantification of BERT Diagnosis Generalizability Across Medical Specialties Using Semantic Dataset Distance](https://mlsanity.com/paper/2008.06606/) (2020)

## Abstract (arXiv)

The combined growth of available data and their unstructured nature has received increased interest in natural language processing (NLP) techniques to make value of these data assets since this format is not suitable for statistical analysis. This work presents a systematic literature review of state-of-the-art advances using transformer-based methods on electronic medical records (EMRs) in different NLP tasks. To the best of our knowledge, this work is unique in providing a comprehensive review of research on transformer-based methods for NLP applied to the EMR field. In the initial query, 99 articles were selected from three public databases and filtered into 65 articles for detailed analysis. The papers were analyzed with respect to the business problem, NLP task, models and techniques, availability of datasets, reproducibility of modeling, language, and exchange format. The paper presents some limitations of current research and some recommendations for further research.