# BioLORD: Learning Ontological Representations from Definitions (for Biomedical Concepts and their Textual Descriptions) arXiv: https://arxiv.org/abs/2210.11892 ยท HTML: https://mlsanity.com/paper/2210.11892/ ## Key facts - Cited by 21 papers in the map (data through 14 Oct 2026). - Builds on 20 of its 31 references that are in the map. - First posted on arXiv 21 Oct 2022. ## Builds on (most cited first, 20 of 20) - [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019) - [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018) - [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019) - [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019) - [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020) - [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017) - [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020) - [Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets](https://mlsanity.com/paper/1906.05474/) (2019) - [SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization](https://mlsanity.com/paper/1911.03437/) (2019) - [Self-Alignment Pretraining for Biomedical Entity Representations](https://mlsanity.com/paper/2010.11784/) (2020) - [Lessons from Natural Language Inference in the Clinical Domain](https://mlsanity.com/paper/1808.06752/) (2018) - [MedMentions: A Large Biomedical Corpus Annotated with UMLS Concepts](https://mlsanity.com/paper/1902.09476/) (2019) - [Biomedical Entity Representations with Synonym Marginalization](https://mlsanity.com/paper/2005.00239/) (2020) - [Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing](https://mlsanity.com/paper/2112.07869/) (2021) - [MedSTS: A Resource for Clinical Semantic Textual Similarity](https://mlsanity.com/paper/1808.09397/) (2018) - [Knowledge-Rich Self-Supervision for Biomedical Entity Linking](https://mlsanity.com/paper/2112.07887/) (2021) - [A Hybrid Approach to Measure Semantic Relatedness in Biomedical Concepts](https://mlsanity.com/paper/2101.10196/) (2021) - [Biomedical Entity Linking with Contrastive Context Matching](https://mlsanity.com/paper/2106.07583/) (2021) - [Biomedical Concept Relatedness -- A large EHR-based benchmark](https://mlsanity.com/paper/2010.16218/) (2020) - [Neural sentence embedding models for semantic similarity estimation in the biomedical domain](https://mlsanity.com/paper/2110.15708/) (2021) ## Built on by (most cited first, 21 of 21) - [Benchmarking Retrieval-Augmented Generation for Medicine](https://mlsanity.com/paper/2402.13178/) (2024) - [An Agentic System for Rare Disease Diagnosis with Traceable Reasoning](https://mlsanity.com/paper/2506.20430/) (2025) - [Towards Evaluating and Building Versatile Large Language Models for Medicine](https://mlsanity.com/paper/2408.12547/) (2024) - [Extreme Multi-Label Skill Extraction Training using Large Language Models](https://mlsanity.com/paper/2307.10778/) (2023) - [BioLORD-2023: Semantic Textual Representations Fusing LLM and Clinical Knowledge Graph Insights](https://mlsanity.com/paper/2311.16075/) (2023) - [Automatic Glossary of Clinical Terminology: a Large-Scale Dictionary of Biomedical Definitions Generated from Ontological Knowledge](https://mlsanity.com/paper/2306.00665/) (2023) - [In-Context Learning for Extreme Multi-Label Classification](https://mlsanity.com/paper/2401.12178/) (2024) - [Boosting Adverse Drug Event Normalization on Social Media: General-Purpose Model Initialization and Biomedical Semantic Text Similarity Benefit Zero-Shot Linking in Informal Contexts](https://mlsanity.com/paper/2308.00157/) (2023) - [Evaluation of Language Models in the Medical Context Under Resource-Constrained Settings](https://mlsanity.com/paper/2406.16611/) (2024) - [A Natural Language Processing Approach to Support Biomedical Data Harmonization: Leveraging Large Language Models](https://mlsanity.com/paper/2411.02730/) (2024) - [A Vision-language Framework for Comparative Reasoning in Radiology](https://mlsanity.com/paper/2606.06407/) (2026) - [When the Edit Changes the Patient: Measuring Identity Preservation in Counterfactual Retinal Images](https://mlsanity.com/paper/2608.23024/) (2026) - [Detecting Idiomatic Multiword Expressions in Clinical Terminology using Definition-Based Representation Learning](https://mlsanity.com/paper/2305.06801/) (2023) - [From Text to Treatment Effects: A Meta-Learning Approach to Handling Text-Based Confounding](https://mlsanity.com/paper/2409.15503/) (2024) - [Semantic Alignment of Unimodal Medical Text and Vision Representations](https://mlsanity.com/paper/2503.04478/) (2025) - [InterFeat: A Pipeline for Finding Interesting Scientific Features](https://mlsanity.com/paper/2505.13534/) (2025) - [Language corpora for the Dutch medical domain](https://mlsanity.com/paper/2604.25374/) (2026) - [Transfer Learning for FHIR Questionnaire Terminology Binding](https://mlsanity.com/paper/2606.15449/) (2026) - [OntologyAligner: Ontology-Aligned Retrieval and Hierarchy-Guided Large Language Model Reranking for Biomedical Ontology Normalization](https://mlsanity.com/paper/2609.10055/) (2026) - [EHRAdapt: Adapting Pretrained Language Models to Electronic Health Records with Semantic Priors for Rare Clinical Events](https://mlsanity.com/paper/2609.34007/) (2026) - [SMI: Ontology-Grounded Semantic IDs for Hierarchy-Aware Medical Concept Representation](https://mlsanity.com/paper/2610.18672/) (2026) ## Abstract (arXiv) This work introduces BioLORD, a new pre-training strategy for producing meaningful representations for clinical sentences and biomedical concepts. State-of-the-art methodologies operate by maximizing the similarity in representation of names referring to the same concept, and preventing collapse through contrastive learning. However, because biomedical names are not always self-explanatory, it sometimes results in non-semantic representations. BioLORD overcomes this issue by grounding its concept representations using definitions, as well as short descriptions derived from a multi-relational knowledge graph consisting of biomedical ontologies. Thanks to this grounding, our model produces more semantic concept representations that match more closely the hierarchical structure of ontologies. BioLORD establishes a new state of the art for text similarity on both clinical sentences (MedSTS) and biomedical concepts (MayoSRS).