Cross-Lingual Knowledge Transfer for Clinical Phenotyping
Not on the map. Why?
Abstract (arXiv)
Clinical phenotyping enables the automatic extraction of clinical conditions from patient records, which can be beneficial to doctors and clinics worldwide. However, current state-of-the-art models are mostly applicable to clinical notes written in English. We therefore investigate cross-lingual knowledge transfer strategies to execute this task for clinics that do not use the English language and have a small amount of in-domain data available. We evaluate these strategies for a Greek and a Spanish clinic leveraging clinical notes from different clinical domains such as cardiology, oncology and the ICU. Our results reveal two strategies that outperform the state-of-the-art: Translation-based methods in combination with domain-specific encoders and cross-lingual encoders plus adapters. We find that these strategies perform especially well for classifying rare phenotypes and we advise on which method to prefer in which situation. Our results show that using multilingual data overall improves clinical phenotyping models and can compensate for data sparseness.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 19 of its 36 references that are in the map. First posted on arXiv 3 Aug 2022.
Builds on
19 of 36 references.
References · 19
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2019Cross-lingual Language Model Pretrainingcited by 1,088
- 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
- 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
- 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
- 2020AdapterHub: A Framework for Adapting Transformerscited by 441
- 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
- 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
- 2019BEHRT: Transformer for Electronic Health Recordscited by 258
- 2020Translation Artifacts in Cross-lingual Transfer Learningcited by 95
- 2020Improving Multilingual Models with Language-Clustered Vocabulariescited by 53
- 2021Lightweight Adapter Tuning for Multilingual Speech Translationcited by 53
7 more
- 2021Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integrationcited by 43
- 2020GREEK-BERT: The Greeks visiting Sesame Streetcited by 37
- 2018Clinical Text Classification with Rule-based Features and Knowledge-guided Convolutional Neural Networkscited by 22
- 2019Phenotyping of Clinical Notes with Improved Document Classification Models Using Contextualized Neural Language Modelscited by 20
- 2021Should we Stop Training More Monolingual Models, and Simply Use Machine Translation Instead?cited by 16
- 2021Biomedical and Clinical Language Models for Spanish: On the Benefits of Domain-Specific Pretraining in a Mid-Resource Scenariocited by 11
- 2021Self-Supervised Detection of Contextual Synonyms in a Multi-Class Setting: Phenotype Annotation Use Casecited by 7
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.