mlsanityData to 7 Oct 2026

Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration

arXiv 2102.04110 ↗First posted 8 Feb 2021cs.CLcited by 43

Not on the map. Why?

Abstract (arXiv)

Outcome prediction from clinical text can prevent doctors from overlooking possible risks and help hospitals to plan capacities. We simulate patients at admission time, when decision support can be especially valuable, and contribute a novel admission to discharge task with four common outcome prediction targets: Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction. The ideal system should infer outcomes based on symptoms, pre-conditions and risk factors of a patient. We evaluate the effectiveness of language models to handle this scenario and propose clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources. We further present a simple method to incorporate ICD code hierarchy into the models. We show that our approach improves performance on the outcome tasks against several baselines. A detailed analysis reveals further strengths of the model, including transferability, but also weaknesses such as handling of vital values and inconsistencies in the underlying data.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 26 of its 48 references that are in the map. First posted on arXiv 8 Feb 2021.

Builds on

26 of 48 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Longformer: The Long-Document Transformercited by 3,362
  3. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  4. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  5. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  6. 2019Publicly Available Clinical BERT Embeddingscited by 995
  7. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  8. 2019ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmissioncited by 487
  9. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  10. 2017Generating Multi-label Discrete Patient Records using Generative Adversarial Networkscited by 197
  11. 2019Do NLP Models Know Numbers? Probing Numeracy in Embeddingscited by 157
  12. 2019A Question-Entailment Approach to Question Answeringcited by 133
14 more
  1. 2018MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcarecited by 72
  2. 2019How Does BERT Answer Questions? A Layer-Wise Analysis of Transformer Representationscited by 63
  3. 2019SECTOR: A Neural Model for Coherent Topic Segmentation and Classificationcited by 48
  4. 2019Using Clinical Notes with Time Series Data for ICU Managementcited by 38
  5. 2020BERT-XML: Large Scale Automated ICD Coding Using BERT Pretrainingcited by 37
  6. 2017Towards Automated ICD Coding Using Deep Learningcited by 35
  7. 2018Learning Tasks for Multitask Learning: Heterogenous Patient Populations in the ICUcited by 21
  8. 2018Patient representation learning and interpretable evaluation using clinical notescited by 15
  9. 2018Deep EHR: Chronic Disease Prediction Using Medical Notescited by 15
  10. 2020Experimental Evaluation and Development of a Silver-Standard for the MIMIC-III Clinical Coding Datasetcited by 15
  11. 2019Towards unstructured mortality prediction with free-text clinical notescited by 8
  12. 2020Self-Attention Enhanced Patient Journey Understanding in Healthcare Systemcited by 7
  13. 2019An Analysis of Attention over Clinical Notes for Predictive Taskscited by 4
  14. 2020Predicting Multiple ICD-10 Codes from Brazilian-Portuguese Clinical Notescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 8; 2023: 10; 2024: 5; 2025: 12; 2026: 2.

43 papers.

Sort:

By area:

  1. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  2. 2023BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Taskscited by 201
  3. 2023Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domaincited by 38
  4. 2023BioInstruct: Instruction Tuning of Large Language Models for Biomedical Natural Language Processingcited by 25
  5. 2024RAM-EHR: Retrieval Augmentation Meets Clinical Predictions on Electronic Health Recordscited by 21
  6. 2021Literature-Augmented Clinical Outcome Predictioncited by 20
  7. 2022Clinical Prompt Learning with Frozen Language Modelscited by 19
  8. 2022Multi-label Few-shot ICD Coding as Autoregressive Generation with Promptcited by 16
  9. 2023NLP for Maternal Healthcare: Perspectives and Guiding Principles in the Age of LLMscited by 14
  10. 2024De-identification is not enough: a comparison between de-identified and synthetic clinical notescited by 14
  11. 2023Auditing and Generating Synthetic Data with Controllable Trust Trade-offscited by 13
  12. 2022On the Importance of Clinical Notes in Multi-modal Learning for EHR Datacited by 10
31 more
  1. 2025Why Chain of Thought Fails in Clinical Text Understandingcited by 8
  2. 2024Generalist embedding models are better at short-context clinical semantic search than specialized embedding modelscited by 7
  3. 2021Detect and Classify -- Joint Span Detection and Classification for Health Outcomescited by 6
  4. 2021What Do You See in this Patient? Behavioral Testing of Clinical NLP Modelscited by 6
  5. 2022This Patient Looks Like That Patient: Prototypical Networks for Interpretable Diagnosis Prediction from Clinical Textcited by 6
  6. 2022Using Bottleneck Adapters to Identify Cancer in Clinical Notes under Low-Resource Constraintscited by 5
  7. 2023Unifying Structure Reasoning and Language Model Pre-training for Complex Reasoningcited by 4
  8. 2024Efficiency at Scale: Investigating the Performance of Diminutive Language Models in Clinical Taskscited by 4
  9. 2025Explainable AI for Clinical Outcome Prediction: A Survey of Clinician Perceptions and Preferencescited by 4
  10. 2021Quantifying Explainability in NLP and Analyzing Algorithms for Performance-Explainability Tradeoffcited by 3
  11. 2023BLIAM: Literature-based Data Synthesis for Synergistic Drug Combination Predictioncited by 3
  12. 2025CliniQ: A Multi-faceted Benchmark for Electronic Health Record Retrieval with Semantic Match Assessmentcited by 3
  13. 2021Patient Outcome and Zero-shot Diagnosis Prediction with Hypernetwork-guided Multitask Learningcited by 2
  14. 2021Assessing gender bias in medical and scientific masked language models with StereoSetcited by 2
  15. 2022Assessment of contextualised representations in detecting outcome phrases in clinical trialscited by 2
  16. 2022Cross-Lingual Knowledge Transfer for Clinical Phenotypingcited by 2
  17. 2023Soft-prompt tuning to predict lung cancer using primary care free-text Dutch medical notescited by 2
  18. 2025DeVisE: Behavioral Testing of Medical Large Language Modelscited by 2
  19. 2025Predicting life satisfaction using machine learning and explainable AIcited by 2
  20. 2024Bespoke Large Language Models for Digital Triage Assistance in Mental Health Carecited by 1
  21. 2025CliniBench: A Clinical Outcome Prediction Benchmark for Generative and Encoder-Based Language Modelscited by 1
  22. 2025SoftAdaClip: A Smooth Clipping Strategy for Fair and Private Model Trainingcited by 1
  23. 2025ClinStructor: AI-Powered Structuring of Unstructured Clinical Textscited by 1
  24. 2022GWO-FI: A novel machine learning framework by combining Gray Wolf Optimizer and Frequent Itemsets to diagnose and investigate effective factors on In-Hospital Mortality and Length of Stay among Kermanshahian Cardiovascular Disease patientscited by 0
  25. 2023On Preserving the Knowledge of Long Clinical Textscited by 0
  26. 2025CKD-EHR:Clinical Knowledge Distillation for Electronic Health Recordscited by 0
  27. 2025BTW: A Non-Parametric Variance Stabilization Framework for Multimodal Model Integrationcited by 0
  28. 2025Integrating Text and Time-Series into (Large) Language Models to Predict Medical Outcomescited by 0
  29. 2025Additive Large Language Models for Semi-Structured Textcited by 0
  30. 2026Enhancing Extubation Failure Prediction with LLM-Derived Features from Respiratory Therapy Clinical Notescited by 0
  31. 2026DP-IPI: A Hybrid Differential Privacy Text Rewriting Mechanism for Indirect Personal Identifiers in Clinical Textscited by 0