# Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking

arXiv: https://arxiv.org/abs/2105.14398 · HTML: https://mlsanity.com/paper/2105.14398/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 30 references that are in the map.
- First posted on arXiv 30 May 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning](https://mlsanity.com/paper/1904.06627/) (2019)
- [Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets](https://mlsanity.com/paper/1906.05474/) (2019)
- [Self-Alignment Pretraining for Biomedical Entity Representations](https://mlsanity.com/paper/2010.11784/) (2020)
- [SenseBERT: Driving Some Sense into BERT](https://mlsanity.com/paper/1908.05646/) (2019)
- [Biomedical Entity Representations with Synonym Marginalization](https://mlsanity.com/paper/2005.00239/) (2020)
- [COMETA: A Corpus for Medical Entity Linking in the Social Media](https://mlsanity.com/paper/2010.03295/) (2020)
- [A Comparison of Word Embeddings for the Biomedical Natural Language Processing](https://mlsanity.com/paper/1802.00400/) (2018)
- [Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity](https://mlsanity.com/paper/1909.02339/) (2019)

## Built on by (most cited first, 28 of 28)

- [Large Language Models and Knowledge Graphs: Opportunities and Challenges](https://mlsanity.com/paper/2308.06374/) (2023)
- [BERT might be Overkill: A Tiny but Effective Biomedical Entity Linker based on Residual Convolutional Neural Networks](https://mlsanity.com/paper/2109.02237/) (2021)
- [Prix-LM: Pretraining for Multilingual Knowledge Base Construction](https://mlsanity.com/paper/2110.08443/) (2021)
- [Overview of BioASQ 2025: The Thirteenth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering](https://mlsanity.com/paper/2508.20554/) (2025)
- [Beyond Static Models and Test Sets: Benchmarking the Potential of Pre-trained Models Across Tasks and Languages](https://mlsanity.com/paper/2205.06356/) (2022)
- [AutoPCR: Automated Phenotype Concept Recognition by Prompting](https://mlsanity.com/paper/2507.19315/) (2025)
- [Collective Human Opinions in Semantic Textual Similarity](https://mlsanity.com/paper/2308.04114/) (2023)
- [Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders](https://mlsanity.com/paper/2205.00267/) (2022)
- [BioLORD-2023: Semantic Textual Representations Fusing LLM and Clinical Knowledge Graph Insights](https://mlsanity.com/paper/2311.16075/) (2023)
- [Biomedical Named Entity Recognition via Dictionary-based Synonym Generalization](https://mlsanity.com/paper/2305.13066/) (2023)
- [Evaluating end-to-end entity linking on domain-specific knowledge bases: Learning about ancient technologies from museum collections](https://mlsanity.com/paper/2305.14588/) (2023)
- [xMEN: A Modular Toolkit for Cross-Lingual Medical Entity Normalization](https://mlsanity.com/paper/2310.11275/) (2023)
- [ClinLinker: Medical Entity Linking of Clinical Concept Mentions in Spanish](https://mlsanity.com/paper/2404.06367/) (2024)
- [BALI: Enhancing Biomedical Language Representations through Knowledge Graph and Language Model Alignment](https://mlsanity.com/paper/2509.07588/) (2025)
- [UMLS-KGI-BERT: Data-Centric Knowledge Integration in Transformers for Biomedical Entity Recognition](https://mlsanity.com/paper/2307.11170/) (2023)
- [RuCCoD: Towards Automated ICD Coding in Russian](https://mlsanity.com/paper/2502.21263/) (2025)
- [SynCABEL: Synthetic Contextualized Augmentation for Biomedical Entity Linking](https://mlsanity.com/paper/2601.19667/) (2026)
- [Beyond English benchmarks: clinical llm evaluation in Brazilian Portuguese](https://mlsanity.com/paper/2606.07853/) (2026)
- [Combining Contrastive Learning and Knowledge Graph Embeddings to develop medical word embeddings for the Italian language](https://mlsanity.com/paper/2211.05035/) (2022)
- [Self-supervised vision-langage alignment of deep learning representations for bone X-rays analysis](https://mlsanity.com/paper/2405.08932/) (2024)
- [CLIMATELI: Evaluating Entity Linking on Climate Change Data](https://mlsanity.com/paper/2406.16732/) (2024)
- [Ontology-Based Concept Distillation for Radiology Report Retrieval and Labeling](https://mlsanity.com/paper/2508.19915/) (2025)
- [BIBERT-Pipe on Biomedical Nested Named Entity Linking at BioASQ 2025](https://mlsanity.com/paper/2509.09725/) (2025)
- [Team Fusion@ SU@ BC8 SympTEMIST track: transformer-based approach for symptom recognition and linking](https://mlsanity.com/paper/2604.06424/) (2026)
- [LongBEL: Long-Context and Document-Consistent Biomedical Entity Linking](https://mlsanity.com/paper/2605.13451/) (2026)
- [BioELX: Context-Aware Cross-lingual Biomedical Entity Linking without Task-Specific Supervision](https://mlsanity.com/paper/2605.27380/) (2026)
- [HCOE: Hyperbolic Clinical Ontology Embeddings from Biomedical Language Models](https://mlsanity.com/paper/2609.30763/) (2026)
- [SMI: Ontology-Grounded Semantic IDs for Hierarchy-Aware Medical Concept Representation](https://mlsanity.com/paper/2610.18672/) (2026)

## Abstract (arXiv)

Injecting external domain-specific knowledge (e.g., UMLS) into pretrained language models (LMs) advances their capability to handle specialised in-domain tasks such as biomedical entity linking (BEL). However, such abundant expert knowledge is available only for a handful of languages (e.g., English). In this work, by proposing a novel cross-lingual biomedical entity linking task (XL-BEL) and establishing a new XL-BEL benchmark spanning 10 typologically diverse languages, we first investigate the ability of standard knowledge-agnostic as well as knowledge-enhanced monolingual and multilingual LMs beyond the standard monolingual English BEL task. The scores indicate large gaps to English performance. We then address the challenge of transferring domain-specific knowledge in resource-rich languages to resource-poor ones. To this end, we propose and evaluate a series of cross-lingual transfer methods for the XL-BEL task, and demonstrate that general-domain bitext helps propagate the available English knowledge to languages with little to no in-domain data. Remarkably, we show that our proposed domain-specific transfer methods yield consistent gains across all target languages, sometimes up to 20 Precision@1 points, without any in-domain knowledge in the target language, and without any in-domain parallel data.