mlsanityData to 7 Oct 2026

ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission

arXiv 1904.05342 ↗First posted 10 Apr 2019cs.CL cs.LGcited by 487

Not on the map. Why?

Abstract (arXiv)

Clinical notes contain information about patients that goes beyond structured data like lab values and medications. However, clinical notes have been underused relative to structured data, because notes are high-dimensional and sparse. This work develops and evaluates representations of clinical notes using bidirectional transformers (ClinicalBERT). ClinicalBERT uncovers high-quality relationships between medical concepts as judged by humans. ClinicalBert outperforms baselines on 30-day hospital readmission prediction using both discharge summaries and the first few days of notes in the intensive care unit. Code and model parameters are available.

Key facts. Cited by 487 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 47; 2022: 62; 2023: 93; 2024: 93; 2025: 127; 2026: 65.

487 papers.

Sort:

By area:

  1. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  2. 2023BloombergGPT: A Large Language Model for Financecited by 923
  3. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  4. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  5. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 195
  6. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  7. 2023A Study of Generative Large Language Model for Medical Research and Healthcarecited by 160
  8. 2021MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcarecited by 159
  9. 2023Large language models in healthcare and medical domain: A reviewcited by 126
  10. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  11. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  12. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 96
38 more
  1. 2021Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?cited by 91
  2. 2023Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Datasetcited by 90
  3. 2021Pre-trained Language Models in Biomedical Domain: A Systematic Surveycited by 87
  4. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 86
  5. 2022Adapting Pretrained Vision-Language Foundational Models to Medical Imaging Domainscited by 85
  6. 2021MS2: Multi-Document Summarization of Medical Studiescited by 80
  7. 2023Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lecturescited by 68
  8. 2022Clinical-Longformer and Clinical-BigBird: Transformers for long clinical sequencescited by 67
  9. 2022Recovering Private Text in Federated Learning of Language Modelscited by 67
  10. 2024Empowering Time Series Analysis with Large Language Models: A Surveycited by 60
  11. 2023MultiLegalPile: A 689GB Multilingual Legal Corpuscited by 59
  12. 2023LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Modelscited by 58
  13. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  14. 2021Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Mediacited by 51
  15. 2022POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detectioncited by 50
  16. 2021HINT: Hierarchical Interaction Network for Trial Outcome Prediction Leveraging Web Datacited by 49
  17. 2022A Multimodal Transformer: Fusing Clinical Notes with Structured EHR Data for Interpretable In-Hospital Mortality Predictioncited by 48
  18. 2023EcomGPT: Instruction-tuning Large Language Models with Chain-of-Task Tasks for E-commercecited by 48
  19. 2022PanGu-Coder: Program Synthesis with Function-Level Language Modelingcited by 47
  20. 2021Biomedical Question Answering: A Survey of Approaches and Challengescited by 46
  21. 2021Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integrationcited by 43
  22. 2023Advancing Surgical VQA with Scene Graph Knowledgecited by 41
  23. 2024HecVL: Hierarchical Video-Language Pretraining for Zero-shot Surgical Phase Recognitioncited by 41
  24. 2023ChatGPT for Shaping the Future of Dentistry: The Potential of Multi-Modal Large Language Modelcited by 40
  25. 2021Does the Magic of BERT Apply to Medical Code Assignment? A Quantitative Studycited by 39
  26. 2024Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentationcited by 38
  27. 2021MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Educationcited by 37
  28. 2022Deep Multi-modal Fusion of Image and Non-image Data in Disease Diagnosis and Prognosis: A Reviewcited by 37
  29. 2021Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Reviewcited by 35
  30. 2023A Comparative Study of Pretrained Language Models for Long Clinical Textcited by 35
  31. 2021Efficient Domain Adaptation of Language Models via Adaptive Tokenizationcited by 34
  32. 2023Mixture-of-Domain-Adapters: Decoupling and Injecting Domain Knowledge to Pre-trained Language Models Memoriescited by 34
  33. 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
  34. 2022Signature Methods in Machine Learningcited by 33
  35. 2022Improving Medical Predictions by Irregular Multimodal Electronic Health Records Modelingcited by 33
  36. 2023GeoGalactica: A Scientific Large Language Model in Geosciencecited by 32
  37. 2024Generative AI for Synthetic Data Generation: Methods, Challenges and the Futurecited by 31
  38. 2023Biomedical knowledge graph-optimized prompt generation for large language modelscited by 30

50 most cited of 487 All 487 (JSON)