mlsanityData to 14 Oct 2026

Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Review

arXiv 2107.02975 ↗First posted 7 Jul 2021cs.CL cs.AIcited by 35

Not on the map. Why?

Abstract (arXiv)

Electronic health records (EHRs), digital collections of patient healthcare events and observations, are ubiquitous in medicine and critical to healthcare delivery, operations, and research. Despite this central role, EHRs are notoriously difficult to process automatically. Well over half of the information stored within EHRs is in the form of unstructured text (e.g. provider notes, operation reports) and remains largely untapped for secondary use. Recently, however, newer neural network and deep learning approaches to Natural Language Processing (NLP) have made considerable advances, outperforming traditional statistical and rule-based systems on a variety of tasks. In this survey paper, we summarize current neural NLP methods for EHR applications. We focus on a broad scope of tasks, namely, classification and prediction, word embeddings, extraction, generation, and other topics such as question answering, phenotyping, knowledge graphs, medical dialogue, multilinguality, interpretability, etc.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 138 of its 301 references that are in the map. First posted on arXiv 7 Jul 2021.

Builds on

138 of 301 references.

References · 138

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  9. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  10. 2020Longformer: The Long-Document Transformercited by 3,413
  11. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  12. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
38 more
  1. 2018Deep contextualized word representationscited by 2,290
  2. 2017Improved Training of Wasserstein GANscited by 2,235
  3. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  4. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  5. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  6. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  7. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  8. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
  9. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,127
  10. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  11. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  12. 2016Bag of Tricks for Efficient Text Classificationcited by 876
  13. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
  14. 2014Distributed Representations of Sentences and Documentscited by 678
  15. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 597
  16. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 546
  17. 2019ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmissioncited by 493
  18. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
  19. 2019Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Modelcited by 431
  20. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 373
  21. 2020Generating Radiology Reports via Memory-driven Transformercited by 368
  22. 2016RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanismcited by 349
  23. 2020CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTcited by 337
  24. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 299
  25. 2019BEHRT: Transformer for Electronic Health Recordscited by 263
  26. 2019ScispaCy: Fast and Robust Models for Biomedical Natural Language Processingcited by 244
  27. 2018Explainable Prediction of Medical Codes from Clinical Textcited by 234
  28. 2015Doctor AI: Predicting Clinical Events via Recurrent Neural Networkscited by 208
  29. 2017Generating Multi-label Discrete Patient Records using Generative Adversarial Networkscited by 197
  30. 2020CORD-19: The COVID-19 Open Research Datasetcited by 180
  31. 2018Visual Interpretability for Deep Learning: a Surveycited by 155
  32. 2017Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysiscited by 148
  33. 2016Recurrent Neural Network for Text Classification with Multi-Task Learningcited by 144
  34. 2020Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generationcited by 141
  35. 2019A Question-Entailment Approach to Question Answeringcited by 134
  36. 2019MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-IIIcited by 122
  37. 2019HEAD-QA: A Healthcare Dataset for Complex Reasoningcited by 101
  38. 2016Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequencescited by 99

50 most cited of 138 All 138 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 3; 2023: 9; 2024: 8; 2025: 12; 2026: 2.

35 papers.

Sort:

By area:

  1. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  2. 2024KG-Rank: Enhancing Large Language Models for Medical QA with Knowledge Graphs and Ranking Techniquescited by 37
  3. 2022A Unified Review of Deep Learning for Automated Medical Codingcited by 26
  4. 2024Enhancing Supply Chain Visibility with Knowledge Graphs and Large Language Modelscited by 11
  5. 2025Explainability in Practice: A Survey of Explainable NLP Across Various Domainscited by 11
  6. 2023Integrating UMLS Knowledge into Large Language Models for Medical Question Answeringcited by 10
  7. 2024A Review on Generative AI Models for Synthetic Medical Text, Time Series, and Longitudinal Datacited by 10
  8. 2025Clinical NLP with Attention-Based Deep Learning for Multi-Disease Predictioncited by 7
  9. 2023PULSAR: Pre-training with Extracted Healthcare Terms for Summarising Patients' Problems and Data Augmentation with Black-box Large Language Modelscited by 6
  10. 2024A Large Language Model and Denoising Diffusion Framework for Targeted Design of Microstructures with Commands in Natural Languagecited by 6
  11. 2025An evaluation of DeepSeek Models in Biomedical Natural Language Processingcited by 6
  12. 2023Linking Surface Facts to Large-Scale Knowledge Graphscited by 5
23 more
  1. 2024The Case for Globalizing Fairness: A Mixed Methods Study on Colonialism, AI, and Health in Africacited by 5
  2. 2023Conceptualizing Machine Learning for Dynamic Information Retrieval of Electronic Health Record Notescited by 4
  3. 2023Ascle: A Python Natural Language Processing Toolkit for Medical Text Generationcited by 3
  4. 2025Predicting Failures of LLMs to Link Biomedical Ontology Terms to Identifiers Evidence Across Models and Ontologiescited by 3
  5. 2021Patient Outcome and Zero-shot Diagnosis Prediction with Hypernetwork-guided Multitask Learningcited by 2
  6. 2025Fact or Guesswork? Evaluating Large Language Models' Medical Knowledge with Structured One-Hop Judgmentscited by 2
  7. 2025Retrieval-Augmented Framework for LLM-Based Clinical Decision Supportcited by 2
  8. 2022EHRKit: A Python Natural Language Processing Toolkit for Electronic Health Record Textscited by 1
  9. 2022Towards Neural Numeric-To-Text Generation From Temporal Personal Health Datacited by 1
  10. 2023UMLS-KGI-BERT: Data-Centric Knowledge Integration in Transformers for Biomedical Entity Recognitioncited by 1
  11. 2023Coupling Symbolic Reasoning with Language Modeling for Efficient Longitudinal Understanding of Unstructured Electronic Medical Recordscited by 1
  12. 2025A Federated Random Forest Solution for Secure Distributed Machine Learningcited by 1
  13. 2025OpenMed NER: Open-Source, Domain-Adapted State-of-the-Art Transformers for Biomedical NER Across 12 Public Datasetscited by 1
  14. 2023Globalizing Fairness Attributes in Machine Learning: A Case Study on Health in Africacited by 0
  15. 2023Can-SAVE: Deploying Low-Cost and Population-Scale Cancer Screening via Survival Analysis Variables and EHRcited by 0
  16. 2024Cross-Institutional Dental EHR Entity Extraction via Generative AI and Synthetic Notescited by 0
  17. 2024Preserving Empirical Probabilities in BERT for Small-sample Clinical Entity Recognitioncited by 0
  18. 2025Trust and Dependability in Blockchain & AI Based MedIoT Applications: Research Challenges and Future Directionscited by 0
  19. 2025Survey: Understand the challenges of MachineLearning Experts using Named EntityRecognition Toolscited by 0
  20. 2025The Evolving Landscape of Generative Large Language Models and Traditional Natural Language Processing in Medicinecited by 0
  21. 2025The Open Syndrome Definitioncited by 0
  22. 2026LLM-Enabled Multi-Agent Systems: Empirical Evaluation and Insights into Emerging Design Patterns & Paradigmscited by 0
  23. 2026Specialty-Specific Medical Language Model for Immune-Mediated Diseasescited by 0